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Both  issues  are  addressed  here  by  defining  traffic  classes,  and  assign¬ 
ing  delay  costs  c(b,t)  which  reflect  the  undesirability  of  a  class-b  packet 
spending  time  t  in  the  network.  We  argue  that  convex,  nondecreasing  (CND) 
functions  are  suitable  for  a  variety  of  classes;  examples  suggest  a  range 
from  linear  (e.g.  file  transfers)  to  step-like  (e.g.  packetized  voice).  An 
appropriate  performance  measure  is  the  limiting  expected  average  cost  per 
packet.  (As  an  equivalent  characterization,  we  show  that,  for  a  given  class, 
the  expected  average  cost  converges  to  the  expected  cost  in  equilibrium, 
under  mild  conditions.)  Within  a  given  class,  strictly  convex  costs  will  be 
driven  by  packets  whose  routes  have  many  hops,  and/or  packets  which  have  long 
delays  on  some  links;’  such  packets  may  profitably  be  given  priority  as  well. 

We  propose  a  distributed  scheme  in  which  a  single  scheduling  policy  is 
executed  independently  on  every  link.  Link  traffic  is  modeled  by  a 
stochastic  sequence' {y^I,  where  <a  »t>n,h  ','r  jxn>  £  ftY  =  IRxBx[0,<»)xRx(0,®) 

represents  for  packet . n  the  epoch  of  arrival  to  the  link  '(a^ca  j),  traffic 

class,  delay  on  upstream  links,  route,  and  transmission  time;  yn(cu)  becomes 

known  to  the  link  scheduler  at  a  (a) .  Routing-scheduling  interactions  and 
communication  overhead  are  mentiSned,  and  desirable  properties  for  estimates 
f  of  the  del  ay .  on.-  downstream  links  are  discussed.  We  develop  a  dynamic 
programming  formulation  in  which  the  scheduler  incurs  a  decision  cost 
g(t,y)  =  c(b,h+t-a*T+f)  by  selecting  for- -transmission  at  epoch  t  a  packet 
with  characteristics  yeY. 

We  focus  on  priority  rules  <J>(t,y):  at  epoch  t,  a  packet  for  which  <J>  is 
maximum  is  selected  for  transmission.  A  rule  <}>*  is  optimal  (in  the  static 
sense)  if  it  empties  out  any  given  queue  at  minimum  cost;  e.g.  for  linear 
costs  g(t,y)  =  ayt,  we  shown  that  the  "yC  rule"  $  =  ny/t  is  optimal  in  this 

sense.  A  significant  result  here  is  that,  in  the  case  of  constant  transmis¬ 
sion  times,  an  optimal  time- independent  rule  minimizes  the  limiting-average 
cost  for  every  w;  thus  for  linear  decision  costs,  the  rule  <f>  =  my  generates 

an  optimal  policy  for  a  G/D/l  link.  For  delay  costs  of  the  form  c(b,t)  = 
cQ(t+A^),  where  cQ  is  CND,  we  show  similarly  that  the  rule  <J>  =  A^+h-a+f  is 

optimal  for  a  G/D/l  link  if  f  is  time-independent.  Suboptimal  rules  are  also 
given  for  certain  costs  in  the  case  of  variable  service  times. 
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ABSTRACT 

Delay  is  a  fundamental  issue  in  packet  communication  networks.  Previ¬ 
ous  work  has  focused  on  expected  delay  as  a  performance  measure  in  queue¬ 
ing  models,  in  the  context  of  routing.  It  is  clear  that  some  types  of  traffic 
are  more  sensitive  to  delay  than  others,  and  some  simple  priority  structures 
have  been  proposed;  but  the  fact  that  expected  delay  is  not  very  meaningful 
for  certain  applications  Ifi.g.  packetized  voice,  which  has  a  critical  delay 
ceiling)  has  received  little  attention. 

Both  issues  are  addressed  here  by  defining  traffic  classes ,  and  assigning 
delay  costs  c(b,t)  which  reflect  the  undesirability  of  a  class-b  packet  spending 
time  t  in  the  network.  We  argue  that  convex,  nondecreasing  (CND)  functions 
are  suitable  for  a  variety  of  classes;  examples  suggest  a  range  from  linear 
{e.g.  file  transfers)  to  step-like  {e.g.  packetized  voice).  An  appropriete  perfor¬ 
mance  measure  is  the  limiting  expected  average  cost  per  packet.  (As  an 
equivalent  characterization,  we  show  that,  for  a  given  class,  the  expected  aver¬ 
age  cost  converges  to  the  expected  cost  in  equilibrium,  under  mild  conditions.) 
Within  a  given  class,  strictly  convex  costs  will  be  driven  by  packets  whose 
routes  have  many  hops,  and/or  packets  which  have  long  delays  on  some  links; 
such  packets  may  profitably  be  given  priority  as  well. 

We  propose  a  distributed  scheme  in  which  a  single  scheduling  policy  is 
executed  independently  on  every  link.  Link  traffic  is  modeled  by  a  stochastic 
sequence  {yn},  where  <an,bn,hf„rn,rn>  •  yn:Q-»Y  ■  IR*B*  [0,<e><  R*(0,«)  represents 
for  packet  n  the  epoch  of  arrival  to  the  link  (an£an«i),  traffic  class,  delay  on 
upstream  links,  route,  and  transmission  time;  y^tw)  becomes  known  to  the  link 
scheduler  at  an(«).  Routing-scheduling  interactions  and  communication  overhead 
are  mentioned,  and  desirable  properties  for  estimates  f  of  the  delay  on 
downstream  links  are  discussed.  We  develop  a  dynamic  programming  formu¬ 
lation  in  which  the  scheduler  incurs  a  decision  cost  g(t,y)  ■  c(b, h  +  t-  a  +  r  +  f)  by 
selecting  for  transmission  at  epoch  t  a  packet  with  characteristics  y  t  Y. 

We  focus  on  priority  rules  ^(t.y):  at  epoch  t,  a  packet  for  which  f  is 
maximum  is  selected  for  transmission.  A  rule  f ’  is  optimal  (in  the  static 
sense)  if  it  empties  out  any  given  queue  at  minimum  cost;  e.g.  for  linear. costs 
g(t,y)  »  myt,  we  show  that  the  '>C  rule"  f  >  my/r  is  optimal  in  this  sense.  A 
significant  result  here  is  that,  in  the  case  of  constant  transmission  times,  an 
optimal  time-independent  rule  minimizes  the  limiting-average  cost  for  every  mt 
thus  for  linear  decision  costs,  the  rule  f  »  my  generates  an  optimal  policy  for 
a  G/D/1  link.  For  delay  costs  of  the  form  c(b,t)  >  c^t  +  Au),  where  c0  is  CND, 
we  show  similarly  that  the  rule  f  «  Ab+h-a+f  is  optimal  for  a  G/D/1  link  if  f 
is  time-independent.  Suboptimai  rules  are  also  given  for  certain  costs  in  the 
case  of  variable  service  times. 
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CHAPTER  I  -  INTRODUCTION 


1.1  Communication  Delays  in  Packet  Networks 

in  many  types  of  communication  systems,  the  traffic  load  is  not  a  static 
quantity;  even  during  periods  of  peak  average  loading,  the  instantaneous  load 
typically  has  a  substantial  variation  about  this  average.  However,  providing  a 
system  with  sufficient  capacity  to  accommodate  these  instantaneous  load 
peaks  can  almost  never  be  justified  economically,  and  the  reduced  allocation 
of  system  capacity  may  result  in  degradation  of  one  or  more  aspects  of 
service  to  the  users. 

In  packet-switched  systems,  such  as  datagram  or  virtual-circuit  communi¬ 
cation  networks,  point-to-point  transmission  of  data  packets  is  accomplished 
over  a  single  shared  channel,  and  packets  reedy  to  depart  are  queued  in  a 
buffer  when  the  transmitter  on  a  link  is  busy  [Tanenbaum;  $3.3.5]  [Heart  et 
a/].  Due  to  the  irregular  pattern  in  which  data  traffic  arrives  from  outside  the 
network,  a  certain  degree  of  queueing  due  to  entering  traffic  will  be  almost 
inevitable,  and  this  will  be  compounded  by  internal  traffic.  In  this  case,  then, 
the  limited  system  capacity  manifests  itself  in  the  form  of  queueing  delays. 
Of  course  there  are  other  delays  associated  with  packet  switching:  however, 
packet  processing  and  transmission  delays  are  relatively  small,  and  propagation 
delays  become  significant  only  in  satellite  applications;  in  any  case,  all  of 
these  are  more-or-less  constant  system  parameters  as  far  as  a  given  user-pair 
is  concerned. 

Communication  delay  is  a  fundamental  performance  issue  in  packet 
networks.  As  one  consideration,  many  applications  have  certain  necessary  or 
desirable  response-time  characteristics  for  the  individual  packets.  As  an 
example,  the  perceived  quality  of  a  display  terminal  session  is  degraded  as  the 


round-trip  delay  is  increased;  in  particular,  it  is  tempting  to  speculate  that  this 
kind  of  consideration  will  be  relevant  whenever  the  psychophysical  needs  of 
the  user  come  into  play.  As  another  example,  in  certain  kinds  of  real-time, 
distributed  processing  environments,  the  value  of  information  decreases  as  it 
gets  older. 

As  a  second  consideration,  it  is  often  the  case  that  each  of  the  individual 
packet  delays  contributes  to  the  total  duration  of  a  given  session  or,  more 
specifically,  of  a  given  job.  In  this  context,  the  effects  of  delay  are  most 
noticeable  in  applications  characterized  by  frequent  exchanges  of  short 
messages;  as  the  frequency  of  such  exchanges  increases,  the  delay  becomes  a 
larger  fraction  of  the  total  time  required  to  complete  the  job.  At  the  other 
extreme,  in  a  "one-sided"  data  conversation  such  as  a  file  transfer,  the  sender 
may  have  many  packets  in  the  network  simultaneously,  in  which  case  the 
effect  of  individual  packet  delays  on  total  job  time  will  be  much  weaker.1 

It  appears  that,  for  many  applications,  delay  considerations  can  be 
resolved  into  one  or  both  of  the  basic  types  described  above;  and  moreover, 
that  one  of  these  will  often  dominate  the  other.  For  instance,  in  a  voice 
conversation,  user-related  constraints  on  the  round-trip  response  times  of  indi¬ 
vidual  packets  should  come  into  play  decisively  at  relatively  low  levels  of 
delay;  again,  one  might  suspect  that  this  would  apply  in  those  cases  in  which 
a  human  is  involved  in  a  direct  way  [ cf .  Roberts  ef  a/;  p.  544].  On  the  other 
hand,  in  certain  types  of  machine-machine  interactions,  individual  packet  delays 
would  be  irrelevant,  and  the  total  job  time  will  be  the  dominant  consideration, 
as  Roberts  ef  a/,  [p.  544]  have  observed. 

*As  en  intermediate  case,  in  a  "window"  flow-control  strategy,  the  sender 
is  allowed  no  more  than  M  packets  in  the  network  at  one  time  [see  Gerla  et 
a/,  1980];  the  job  time  will  then  be  determined  essentially  by  the  round-trip 
delays  of  every  Mth  packet. 


The  primary  motivation  for  distinguishing  these  two  types  of  applications 
is  that  they  typically  lead  to  different  kinds  of  performance  criteria,  as 

discussed  in  $2.1.  For  brevity,  applicat  ons  for  which  the  individual  packet 
delays  are  the  dominant  consideration  will  be  referred  to  here  as 
packet- sensitive,  and  those  for  which  the  job  time  dominates  will  be  called 
job-sensitive. 

As  a  final  consideration,  from  the  network  operator's  point  of  view, 

congestion  will  result  if  packets  spend  too  much  time  in  the  network. 
Although  one  could  attempt  to  incorporate  this  into  delay  costs,  we  expect 
that  buffering  capability  will  be  such  that  congestion  is  typically  dominated  by 
other  considerations. 

1.2  Delay  Performance  Criteria 

Published  work  on  packet  networks  has  been  concerned  almost  exclusive-* 
ly  with  reducing  the  average  delay  per  packet.  In  particular,  considerable 

progress  has  been  made  in  the  development  of  packet  routing  algorithms  under 
the  average  delay  criterion,  due  in  part  to  the  linearity  of  this  performance 
measure  in  the  individual  link  delays  [see  e.g.  Gallager]. 

However,  average  delay  as  a  performance  measure  has  two  serious  weak¬ 
nesses.  First  of  all,  desirable  delay  characteristics  for  data  traffic  will  vary 

over  a  wide  range  for  different  applications.  Roberts  et  a/,  [p.  544]  give 

some  figures  for  typical  time-sharing  applications  as  follows:  a  50  character 
line  of  text  (400  bits)  should  traverse  the  network  in  at  most  200ms;  for  inter¬ 
active  graphics,  a  new  display  page  <20  kbits)  should  take  less  than  a  second, 
and  interrupts  (less  than  100  bits)  within  30-90ms.  According  to  Bell  Labs  [p. 
56],  voice  conversation  is  not  "unduly"  impaired  if  the  origin-destination  delay 
is  less  than  300ms.  Low  priority  items,  such  as  certain  kinds  of  file  transfers 
or  sensor  data,  could  reasonably  be  expected  to  tolerate  delays  on  the  order 


of  several  seconds  or  more.  One  recent  analysis  which  addresses  this  issue 
provides  for  scheduling  voice  packets  ahead  of  data  packets  [Ibe]. 

At  a  more  fundamental  level,  however,  for  certain  types  of  applications 
average  delay  simply  is  not  very  meaningful.  This  is  perhaps  best  illustrated 
by  an  example  such  as  the  following:  in  a  packetized  voice  transmission 
system,  voice  samples  or  "frames"  must  be  delivered  to  the  destination  at 
regular  intervals,  to  be  reassembled  and  decoded  for  the  listener;  if  a  packet 
arrives  early  it  is  simply  buffered;  if  it  arrives  late  it  must  be  discarded.  It  is 
fairly  clear  that  the  fraction  of  packets  arriving  "on  time"  is  all  we  really  care 
about  here;  depending  on  the  distribution  of  packet  delays,  the  average  delay 
need  not  be  a  very  reliable  indicator  of  this. 

Thus  the  figures  cited  in  the  next  to  last  paragraph  leave  something  to  be 
desired;  for  instance,  must  we  hold  the  delay  of  every  text  packet  below 
200ms.  or  will  we  be  content  to  achieve  this  as  an  average  value?  More 
specifically,  what  are  the  consequences  of  exceeding  this  value,  and  what,  if 
anything,  do  we  gain  by  undercutting  it?  Such  questions  have  received  little  if 
any  attention  in  the  literature. 

For  certain  aspects  of  telephone  system  performance.  Bell  System  spec¬ 
ifications  are  reported  to  be  established  as  follows.  The  expected  percentage 
of  users  that  would  bestow  a  particular  subjective  rating  (e.g.  "good"),  in 
conjunction  with  a  particular  aspect  of  raw  performance  (e.g.  receiver  rms 
noise  voltage),  is  called  the  grade  of  service  associated  with  that  rating  and 
performance  aspect  [Bell;  pp.  45-48];  the  calculation  of  grades  of  service  is 
based  in  part  on  subjective  testing  of  customers.  Typical  objectives  are  to 
provide  grades  of  service  of  about  95%  in  the  "good  or  better"  category  and 
a  negligible  fraction  in  the  "poor"  category. 

The  grade  of  service  concept  represents  a  step  in  the  right  direction 
because  it  reflects,  in  essence,  a  functional  dependence  of  user  satisfaction  on 


raw  performance.  Here,  however,  we  are  primarily  interested  in  the  automatic 
control  of  delay  performance  in  an  operational  network,  and  this  will  require 
an  objective  function,  /.a.  a  scalar  measure  of  global  performance,  rather  than 
a  collection  of  individual  specifications.  It  is  common  in  an  operations 
research  context  to  assign  costs  as  functions  of  delay  for  various  commod¬ 
ities,  with  the  expected  cost  as  a  performance  measure.  We  adopt  this 
approach  here,  and  thus  exploit  the  notion  of  user  dependence  in  a  very  direct 
way. 

We  are  motivated  here  by  the  philosophy  that,  for  a  given  application, 
any  user  should  be  able  to  expect  a  certain  level  of  performance  from  the 
system,  regardless  of  geographical  separation  or  other  non-essential  factors. 
As  Wong  et  a/,  have  pointed  out,  this  is  particularly  reasonable  when  users  are 
charged  on  the  basis  of  resource  usage.  Our  goal  then  is  to  provide  accepta¬ 
ble  service,  for  all  users  at  as  small  a  dollar  cost  as  possible  to  the  operator, 
or  equivalently  for  as  many  users  as  possible  at  a  given  dollar  expenditure. 

In  Chapter  II.  we  develop  a  cost  structure  for  packet  delays,  and  show 
how  it  addresses  the  issues  raised  here.  A  probabilistic  framework  is  defined, 
and  by  considering  equilibrium  behavior  some  qualitative  insights  are  obtained. 
In  Chapter  III.  we  describe  a  distributed  implementation  of  the  basic  schedul¬ 
ing  idea,  and  construct  a  model  for  the  sequential  decision  process  associated 
with  packet  selection.  We  can  show  that  in  certain  idealized  cases,  optimal 
policies  are  given  by  simple  selection  rules. 


CHAPTER  II  -  A  COST  STRUCTURE  FOR  PACKET  DELAYS 


2.1  Delay  Cost  Functions 

Let  B  be  some  set  of  data  traffic  classes:  for  instance,  B  might  be  a  set 
of  applications,  e.g.  video  terminal,  packetized  voice,  etc;  but  B  is  not 
restricted  to  any  particular  interpretation,  and  we  require  only  that  it  be  a 
countable  set.  We  assume  the  availability  of  a  function  c  from  B*  [0^>)  into 
the  real  numbers  IR,  such  that  c(b,t)  represents  the  undesirability,  from  the 
standpoint  of  the  entity  operating  the  network,  of  a  "typical"  user  experiencing 
an  individual  packet  delay  t  for  a  class-b  packet,  t*0,  beB.  For  short,  the 
function  c(b,  }  is  referred  to  simply  as  the  delay  cost  for  class-b  packets.  It 
isn't  obvious  how  such  functions  would  be  constructed  in  practice.  They 
should  of  course  draw  on  subjective  testing  for  applications  directly  involving 
humans;  it  would  probably  be  most  straightforward  in  such  cases  to  construct 
for  t£0  the  associated  marginal  cost  function1 

m(b.t)  m  ac(b,t)/at,  (2.1) 

i.e.  m(b,t)At  reflects  the  aproximete  cost  of  increasing  the  delay  of  a  class-b 
packet  from  t  to  t+At.  Then  the  deley  cost  clearly  may  be  derived  as 

c(b.t)  «  /  m(b,x)dx  ♦  c(b,0)  (2.2) 

o 

for  some  c(b,0). 

Now  consider  a  collection  of  packets,  labelled  (uniquely)  by  either  the 
positive  integers  IN4  or  a  subset  {1,2, ...N),  which  traverse  some  system; 


backing  the  standard  "equals  by  definition"  symbol  in  our  character  set, 
we  will  make  do  with 


suppose  that  packet  n  belongs  to  class  bneB  and  spends  time  s„£0  in  the 
system.  Obvious  measures  of  performance  are  the  total  cost 

I  c(bn,s n)  (2.3) 

n»  1 

in  the  finite  case,  and  the  long-term  average  cost  per  packet 

lim  (1/N)  J  c(brvSn)  (2.4) 

N—  n‘1 

in  the  infinite  case,  assuming  convergence.  The  additive  structure  without  any 
weighting  implicitly  requires  that  the  delay  costs  e(b,  )  reflect,  as  much  as 
possible,  the  relative  urgency  of  the  various  traffic  classes.  It  also  implies 
that  time-independent  constants  in  the  delay  costs  are  irrelevant  to  the 
minimization  problem,  so  that  we  may  as  well  take  c(b,0)  ■  0  in  (2.2). 

In  an  attempt  to  get  some  feeling  for  the  general  behavior  of  delay  cost 
functions,  we  will  speculate  on  their  form  for  some  representative 
applications.  At  the  same  time,  we  will  argue  that  functions  which  are  convex 
and  nondecreasing  (or  CND  for  short)  are  suitable  as  delay  costs  for  a  reason¬ 
ably  wide  variety  of  applications;  it  turns  out  that  CND  cost  functions  have  a 
number  of  desirable  properties,  and  we  want  to  exploit  these  in  the  sequel. 
We  note  here  that  for  a  convex  cost  c(b,  ),  the  associated  marginal  cost  m(b,  ) 
is  nondecreasing,  and  this  answers  in  a  general  way  the  questions  raised  in 
$1.2  concerning  constraints  on  raw  delay:  we  lose  at  least  as  much  by 
exceeding  a  given  constraint  as  we  would  gain  by  undercutting  it  by  the  same 
amount  [cf.  Haji  et  a/]. 

Consider  first,  as  a  packet-sensitive  application,  the  packetized  voice 
connection  mentioned  earlier  in  $1.2.  The  cost  associated  with  the  transit  time 
of  a  single  voice  packet  is  essentially  a  step  function,  i.e.  of  the  form 


u-Ht)  *  0  for  tST  and  1  for  t>T,  where  T  is  a  threshold  representing  the  desired 
delay  between  arrival  to  the  network  and  decoding  at  the  destination.1  The 
average  cost  (assuming  voice  traffic  only)  is  then 

(1/N)  Z  Ut(s^  (2.5) 

n«  1 

which  is  the  fraction  of  packets  for  which  sn>T. 

Now  in  the  context  of  an  operating  network,  a  desirable  control  mech¬ 
anism  would  be  aware  of  this  threshold,  and  would  attempt  to  ensure  that 
voice  packets  do  not  cross  it.  But  we  can  accomplish  the  same  effect  with  a 
convex,  non-decreasing  function  such  as  c(voice,  )  of  Pig.  1.  The  price  paid  for 
using  the  convex  delay  cost  is  that  the  control  mechanism  will  continue  need¬ 
lessly  to  pay  attention  to  packets  past  the  threshold.  However,  under  normal 
operating  conditions  </.e.  when  the  traffic  load  is  not  excessive),  we  would 
expect  the  fraction  of  packets  exceeding  the  threshold  to  be  very  small  if  the 
control  mechanism  is  effective. 

As  a  second  application,  consider  the  access  of  a  remote  facility  using  a 
display  terminal,  in  terms  of  the  round-trip  response  time.  We  assume  here 
that  processing  in  the  remote  facility  normally  has  a  relatively  small  time 
requirement.  Sufficiently  short  response  times  are  not  objectionable  (or  even 
noticeable)  to  the  typical  user  and  the  delay  cost  will  be  fiat  over  this  range. 
The  typical  user  will  then  become  increasingly  impatient  with  increasing 
response  time  beyond  a  certain  point.  Finally,  the  user  resigns  him  or  herself 
to  the  likelihood  of  an  extraordinary  delay,  at  which  point  the  delay  cost 

*User  satisfaction  will  of  course  depend  also  on  the  value  of  the  thresh¬ 
old  T  itself,  but  the  selection  of  a  threshold  is  a  separate  issue. 

4This  functional  form  for  delay  costs  is  also  similar  to  that  depicted  by 
Lawler  [1964;  Fig.  1]  in  a  job  shop  context. 


levels  off  egein.  A  reasonable  delay  cost  might  look  something  like  the 
dashed  curve4  of  Fig.  2.  The  control  mechanism  should  presumably  try  to  keep 
the  majority  of  packets  below  the  "knee”  of  the  curve  at  T(disp),  and  all  but  a 
negligible  fraction  below  the  "shoulder"  at  T  Jdisp).  Again,  we  can  achieve  a 
similar  effect  with  the  solid  version  c(disp,  )  of  the  curve,  which  is  convex 
and  nondecreasing,  with  the  same  implications  as  in  the  previous  example. 

In  this  case,  the  delay  cost  was  naturally  expressed  in  terms  of  the 
round-trip  response  time.  To  conform  with  our  earlier  definition,  we  might 
now  try  to  find  functions  c(user,  )  and  c(reply,  ),  reflecting  delay  costs  for 
individual  user  and  reply  packets,  which  satisfy 

c(disp,t*s)  *  c(user.t)  ♦  c(reply.s)  (2.6) 

for  all  t  and  s;  but  it  isn't  hard  to  show  that,  because  c(disp,  )  is  nonlinear, 
this  is  impossible.  One  could,  of  course,  seek  a  "best  approximation"  (e.g.  in 
the  sense  of  expected  values  in  the  equation  above),  and  intuitively  we  expect 
that  any  reasonable  cost  functions  so  derived  will  also  be  CND.  At  the  same 
time,  however,  the  approach  that  wilt  be  developed  here  is  in  principle  eppli- 


cable  to  round-trip  delay  costs  as  well;  the  assumption  of  valid  individual 
packet  delay  costs  for  "round-trip-sensitive"  applications  may  be  viewed  as 
essentially  a  notational  simplification. 

For  job-sensitive  applications,  the  dominant  consideration  is  the  total  job 
duration,  which  is  in  general  the  sum  It  <  of  an  unspecified  number  of  compo¬ 
nents.  The  concern  of  the  user  (or  manager)  will  typically  be  the  fact  that 
processing  facilities  and  possibly  personnel  must  be  dedicated  to  the  job  for 
its  duration;  there  may  also  be  connect-time  charges,  both  for  the  network  and 
for  remote  resources  not  owned  by  the  user.  For  a  given  application,  the 
natural  approach  would  again  be  to  express  these  losses  as  a  job  delay  cost 
function  c(job.  ).  In  this  case,  however,  connect-time  charges  will  typically  be 
linear,  and  in  general  a  linear  cost  seems  most  reasonable  for  dedicated 
user-owned  resources;  the  well-known  business  maxim  "time  is  money"  is 
offered  as  a  partial  justification.  Then  we  can  write 

c(job.Zti)  »  mXti  ■  Zmt1t  (2.7] 


Figure  2  Possible  delay  cost  function  for  display  terminal 


so  that  a  linear  cost  c(i.t)  «  mt  {which  is  convex  and  nondecreasing)  is  appro¬ 
priate  for  the  individual  packet  delay  components  as  well. 


2.2  Stochastic  Performance  Measures 

Of  course  the  class  bn  and  system  time  sn  of  packet  n  are  not  known  in 
advance.  In  queueing  and  scheduling  literature  it  is  typical  to  model  the  uncer¬ 
tainty  about  such  a  situation  by  a  stochastic  sequence,  say  fz,*  nc  IN*}  or 
{z„  n«1,...N},  where  zn  is  a  random  "vector"  (or  more  properly  a  random 
ordered  set),  e.g.  <an.br^£n>.  whose  alemants  represent  the  essential  character¬ 
istics  of  packet  n,  e.g.  the  epoch*  an  of  arrival  to  the  system,  class  b^  and 
length  The  r.v.  zn  are  alt  functions  on  an  appropriate  sample  space  0, 
taking  values  in  some  space  Z,  e.g.  IR»B*(0,«);  we  assume  in  particular  that  the 

arrival  epochs  satisfy  an(«)  S  an+i(e)  for  all  ecQ.  The  statistics  of  the  packet 

sequence  (zn)  are  specified  by  a  probability  measure  P.  which  reflects  the 
likelihood  of  various  avents  defined  in  terms  of  the  zn. 

Associated  with  each  point  »eQ  is  a  sample  sequence  or  realization 

{trfr)}  of  the  packet  generation  process;  we  may  than  assign  to  every  such 

realization  a  value  x(«)  from  some  space  X,  in  which  case  the  function  xtfh»X 
is  a  random  variable*  on  0  as  well,  and  its  statistics  are  in  principle  deter¬ 
mined  by  the  probability  measure  P.  We  generally  abbreviate  events  such  as 
{«<Q:  x(«)  <  A}  by  {x(«)cA}  or  just  {xcA},  and  if  P{x<A)  «  1,  we  may  write 
"x«A  w.p.l"  (/.e.  with  probability  one). 


*!n  deference  to  tradition,  a  "time"  will  denote  the  duration  of  a 
(temporal)  event,  whereas  sn  "epoch"  will  denote  a  specific  instant. 

*A  rigorous  construction  would  require  that  x  be  measurable  on  0  [Ash; 
$$1.5,  5.6];  this  ensures  probability  assignments  for  events  defined  in  terms  of 
x  Ws  will  simply  assume  that  the  packet  process  and  system  parameters  are 
such  that  this  requirement  is  met  for  all  functions  on  0  defined  here. 


We  assume  that  the  system  will  treat  packets  in  accordance  with  some 
deterministic  policy  *  (chosen  from  a  given  set  H  of  policies),  so  that  associ¬ 
ated  with  each  realization  {zn(»)}  of  the  external  packet  process  will  be  a 
well-defined  (although  possibly  unbounded)  sequence  {s^(»,*)}  of  nonnegative 
system  times;  i.e.  the  s^l*)  are  random  variables  on  0.  Again,  obvious 
performance  measures  for  the  policy  v  are  the  expected  total  cost 

JN(»)«  2  E{c[b  p,s  „(»)]}  (2.8) 

n«  1 

and  the  lon^g-run  expected  average  cost* 

JM  ■  lim  (1/N)  2  E{c[brysrt(.)l};  (2.9) 

N I  n»  1 

when  the  limit  exists;  thus  we  seek  practical  policies  for  which  JN  or  J  is 
small.  We  will  not  always  make  explicit  the  •  dependence,  but  some  underly¬ 
ing  policy  is  always  assumed. 

For  certain  traffic  classes,  particularly  packet-sensitive  classes,  it  is 
appropriate  to  define  performance  spocificotions  as  well.  We  may  construct 
such  a  specification  for  any  given  class  bcB  by  imposing  a  simple  constraint 
on  the  associated  long-run  cost.  We  will  assume  here  that  the  event 

Ob  *  (•:  {z r(«)}  contains  infinitely  many  class-b  packets)  (2.10) 

’It  might  seem  more  natural  to  consider  instead  the  expectation  of  the 
limit  E{lim(1/N)2NC[bn,sn()r)]},  but  (2.9)  is  more  convenient  from  a  technical 
standpoint;  in  any  case,  if  there  exists  a  random  variable  M  such  that 
(l/N>2Nc[br4»),sH(»,>r)]  <  MM  for  all  «,  N,  then  by  the  dominated  convergence 
theorem  [Ash;  p.  49]  the  two  performance  measures  are  equivalent  for  a  given 
policy  r. 

•We  can  write  P(0-G^  ■  P(lim  inf n  {b,/b});  this  will  be  0  if,  for  instance, 
the  bn  are  independent  and  lim  inf  n  P{bn»b}  >  0  [cf.  Ash  et  »l,  1975;  p.  136]. 
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has  probability  one;*  than  for  every  let 


n<(b,«)  ■  min{N:  IX{bril»)*b}  ■  j} 

n»  1 


(2.11) 


be  the  index  of  the  jth  class-b  packet  in  {zn(«)}.  where  the  indicator  A{bn*b}  a 
1  if  brim)  «  b  and  0  otherwise.  Then  we  can  require  of  the  long-run  expected 
average  cost  per  class-b  packet  that 


J(b)  a  lim  (1/N)  I  E{c[b,s(nj)]  i  c[b,T(b)] 

Ni  n«  1 


(2.12) 


for  some  appropriately  chosen  "target  time"  T(b),  such  as  T(disp)  in  Fig.  2 
(thus  two  traffic  classes  may  have  the  same  delay  cost  but  different  specifica¬ 
tions).  The  additive  nature  of  the  performance  measures  suggests  that  the 
delay  costs  be  individually  scaled  (multiplicatively)  such  that  c[b,T(b)]  is  a 
constant  (say  1)  independent  of  b;  this  makes  precise  the  manner  in  which  the 
delay  costs  should  reflect  the  relative  urgency  of  their  respective  classes. 

One  might  also  want  to  define  one  or  more  'floating"  traffic  classes 
without  specifications.  In  the  network  synthesis  problem  [ef.  Gerla  et  #/, 
1977]  performance  for  these  classes  could  be  traded  off  for  savings  in  dollar 
cost,  while  specifications  for  the  other  classes  are  guaranteed.  In  an  opera¬ 
tional  network,  the  cost  functions  for  the  floating  classes  could  be  multiplica¬ 
tively  scaled  as  a  group,  from  time  to  time  as  necessary,  such  that 
specifications  for  the  constrained  classes  are  just  met. 

Finally,  it  will  also  be  useful  to  have  a  performance  measure  in  terms  of 
the  equilibrium  distribution  of  system  time  associated  with  a  given  class  of 
packets;  the  equilibrium  distribution  is  a  much  more  intuitive  entity  to  think 
about,  and  is  known  for  many  representative  queueing  situations,  Define 


vv\ 


Fj(t;b)  «  P{s[nj{b,*)]St|G 


(2.13) 


the  distribution  of  system  time  for  the  jth  class-b  packet,  where  nj(b,»)  is 
defined  in  (2.11).  Now  suppose  there  exists  a  distribution  function  F(  ;b)  which 
is,  in  some  sense,  an  arbitrarily  close  approximation  to  Fj(  ;b)  for  sufficiently 
large  j;  then  F(  ;b)  corresponds  to  what  we  would  think  of  as  the  distribution 
of  system  times  seen  by  class-b  packets  in  equilibrium.  The  sense  in  which 
F j(  ;b)  is  required  to  approach  F(  ;b)  in  this  context  is  called  weaAr  convergence 
[Ash;  §8.1];  weak  convergence  of  a  sequence  of  probability  distribution  func¬ 
tions,  denoted  here  by  Fn  w-  F,  is  equivalent*  to  pointwise  convergence  Fn(t)  - 
F(t)  for  every  continuity  point  t  of  F  (including  ±«). 

Now  if  Fj(  ;b)  '*•  F(  ;b),  we  might  expect  that,  in  some  cases, 

/c(b.t)dF(t;b)  -  J(b),  (2.14) 

/.e.  that  the  expected  cost  of  every  class*b  packet  arriving  after  the  system  is 
in  equilibrium  is  indicative  of  the  long-run  average  cost.  In  the  case  of  linear 
costs,  certain  system  models  have  been  shown  to  obey  relations  Similar  to 
(2.14)  [Fife]  [Stidham;  p.  1122],  Without  assuming  a  specific  model,  we  can 
show  that  (2.14)  holds  for  every  continuous,  nondecreasing,  nonnegative  delay 
cost  c(b,  ),  under  a  technical  condition  on  the  sequence  {c[b,s(nj)]}  which 
should  be  satisfied  for  most  models  of  interest  (note  that  every  convex  delay 
cost  is  continuous);  a  formal  statement  and  proof  are  given  in  5A.1. 


♦however,  it  is  often  defined  as  follows:  Fn  F  iff  /x(t)dFn(t)  -  /x( t)dF(t) 
for  every  bounded,  continuous  function  x:  IR  -  IR. 


2.3  Delay  Performance  in  Equilibrium 

in  the  light  of  the  previous  paragraph,  we  expect  that  delay  performance 
in  equilibrium  will  be  representative  of  long-run  average  performance  as  well, 
and  we  will  often  tacitly  assume  that  (2.14)  is  true.  We  first  obtain  a  charac¬ 
terization  of  equilibrium  delay  performance  for  a  typical  packet  switching 
network. 

2.3.1  Packet  Network  Model 

Our  model  of  a  packet  network  is  a  set  V  of  nodes,  which  represent 
packet  switches,  and  a  set  E  of  links,  where  link  <i,j>  represents  a  one-way 
transmission  channel  from  node  i  to  node  j;  thus  <V,E>  is  a  (directed)  graph 
[Lawler,  1976;  $2.3].  Associated  with  each  link  is  a  packet  queue  feeding  a 
transmitter,  and  we  assume  unlimited  buffer  space.  The  transmitter  on  link  e 
has  capacity  6  a  bits/sec,  so  that  transmission  of  a  packet  of  random  length  a 
bits  on  link  a  takes  a  random  time  r#  ■  UIB  ,  seconds.  The  structure  just 
described  is  often  called  the  "subnet;"  the  user’s  computer  system,  or  "host," 
is  interfaced  to  one  of  the  subnet  nodes  [Tanenbaum;  $1.2].  We  view  the 
packet  process  {zn}  as  being  generated  by  the  ensemble  of  external  hosts, 
and  the  policy  *  as  a  responsibility  of  the  subnet. 

Packet  n  is  generated  with  a  specified  destination,  and  in  real  life  the 
subnet  must  select  for  it  a  route  t„  i.e.  a  sequence  of  links  from  the  origin 
node  to  the  destination  node.  The  subnet  will  often  have  other  responsibilities 
as  well,  such  as  congestion  control  [Gerla  et  el.  I960].  Here,  however,  we 
want  to  focus  on  subnet  scheduling;  we  first  of  all  want  as  simple  as  possible 
a  structure  for  the  subnet  policy  in  our  model,  so  we  now  make  the  explicit 
qualification  that  >  is  a  subnet  scheduling  policy  only. 

Secondly,  we  are  not  going  to  pursue  in  any  depth  the  affects  of  possi¬ 
ble  interactions  between  scheduling  and  other  subnet  functions.  As  an  exam- 


pte,  define  »0*  "first-come-first-served  (FCFS)  scheduling  on  every  link,"  and 
suppose  »r  is  the  same  except  that  it  always  gives  priority  to  packets  on 
some  route  r.  If  the  route  assignments  are  identical  in  each  case,  it  is  intu¬ 
itively  obvious  (and  very  likely  true  as  well)  that  the  delay  on  route  r  will  be 
smaller  under  »r  than  under  *q.  For  this  reason,  however,  an  adaptive  routing 
strategy  might  assign  a  larger  portion  of  traffic  to  r  under  »r  than  under  »0 
[Tanenbaum;  $5.2], 

For  concreteness,  we  will  assume  that  every  packet  vector  z n(»)  contains 
a  route  rn(»)  in  some  set  R;  the  system  time  snM  is  then  the  sum  of  the 
queueing  and  transmission  times  for  packet  n  as  it  traverses  the  succesive 
links  in  the  route  rj [*).  We  also  implicitly  assume  a  flow  control  mechanism, 
situated  between  the  hosts  and  the  subnet,  which  keeps  link  loading  at  a 
reasonable  level. 

Suppose  now  that,  as  n  **  •,  and  for  every  route  rcR, 

P{sn<t.  rn*r]  H(t,r).  (2.15) 

It  then  follows  that,  as  n  ■*  «, 

P  {r  n*r)  ■  P{SnS«>,  rn«r}  -  H(«»,r)  *  G(r),  (2.16) 

P{snSt}  «  Z  P{s„St,  rn*r}  Z  H(t,r)  ■  F(t),  (2.17) 

R  R 

and,  as  long  as  P  {r  n»r}  ,G(r)  *  0, 

P{8nSt|rn«r}  «  Pjs^t,  rn«r}+P{rn»r)  H(t,r)+G(r)  *  Fr(t).  (2.18) 

Then  F(t)  *  Z|)Fr(t)G(r),  i.e.  the  equilibrium  distribution  of  system  times  is  the 
sum  of  the  distributions  corresponding  to  the  individual  routes  weighted  by  the 
equilibrium  route  probabilities,  as  we  might  have  expected. 


-17- 


By  making  some  simplifying  assumptions,  we  can  invoke  a  standard 
result  to  get  a  qualitative  picture  of  F(t).  It  is  typical  to  model  a  packet 
network  as  a  collection  of  independent  M/G/1  queues,  because  the  distribution 
of  delay  is  known  in  this  case  [Kleinrock.  1975;  ch.  5].  In  particular,  for  an 
M/M/1  queue  in  which  packets  are  transmitted  FCFS,1*  the  density  of  delay 
(queueing  plus  transmission)  on  link  e  has  the  exponential  form  f*(t)  « 
(/>«-Xa)exp  [-</ia-Xw)t] ,  where  /i ,  «  1/E(t J  is  the  service  rate  and  is  the 
arrival  rate  [Kleinrock,  1975;  p.  202].  Moreover,  if  the  link  capacities  0*  are 
such  that  is  independent  of  e,  the  system  time  over  a  given  route 

r  of  j  hops  (links)  has  an  Erlang  density 

fr(t)  «  aFp(t)/et  «  J1tJ"1exp(-4tK(j-i)l,  (2.19) 

expectation  Sr  *  /tdFr(t)  ■  j  It.  and  variance  j  li2  [for  a  picture  see  Kleinrock, 
1975;  Fig.  4.5,  p.  124];  thus  for  such  an  idealized  model  the  overall  equilibrium 
density  f(t)  •  Z„fr(t)G(r)  of  system  times  is  a  weighted  sum  of  the  Erlang 
densities  corresponding  to  the  different  routes,  and  might  look  something  like 
Fig.  3.  While  the  M/M/1  packet  network  model  is  a  fairly  reasonable  one  from 
the  standpoint  of  mean  delay  [Kleinrock,  1976;  ch.  5],  we  do  not  expect  an 
accurate  representation  of  the  entire  distribution;  yet  neither  do  we  expect 
such  limitations  to  materially  alter  the  qualitative  picture  developed  here. 


1 ‘This  model  is  equivalent  to  the  "network  of  Markovian  queues"  [Klein¬ 
rock,  1975;  $4.8]  (in  which  case  the  queues  correspond  to  the  vertices  of  a 
graph  rather  than  the  edges). 
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2.3.2  Implications  of  Convex  Delay  Costs 

For  a  linear  cost  c(t)  *  mt,  we  don't  really  care  about  the  shape  of  f,  we 
just  want  the  equilibrium  mean  S  ■  /tdF(t)  to  be  small.  However,  as  the 
convexity  of  c  becomes  more  pronounced,  the  mean  becomes  less  crucial,  and 
we  generally  become  concerned  more  with  higher  moments  of  delay;  e.g.  if 
c(t)  «  (t/T)k  (T  is  some  threshold),  we  want  the  kth  moment  /t kdF(t)  to  be 
small.  In  turn,  the  higher  moments  are  typically  driven  by  the  upper  tail  of 
the  equilibrium  density,  which  corresponds  to  packets  whose  routes  have  large 
numbers  of  hops,11  and/or  packets  which  have  long  queueing  times  at  some 
links  on  their  routes.  Transmission  scheduling  offers  the  potential  for  mitigat¬ 
ing  both  of  these  contributions  to  the  upper  tail,  by  giving  priority  (at  the 
transmitter)  to  packets  whose  routes  have  large  numbers  of  hops,  and  by 
giving  priority  to  packets  which  have  had  long  queueing  times  at  "upstream" 


</'.».  earlier)  links  in  their  route,  or  are  expected  to  do  so  at  "downstream"  {j.e. 
later)  links. 

We  immediately  qualify  this,  however,  by  noting  that,  in  order  for  sched¬ 
uling  to  be  effective,  there  must  be  packets  to  schedule.  We  should  expect 
that  scheduling  can  be  most  effective  in  environments  charcterized  by  large 
numbers  of  packets  in  queue,  such  as  high-speed  systems.  Scheduling  might 
also  turn  out  to  be  most  useful  as  a  smoothing  response  to  transient  load 
surges. 

We  expect  that  pulling  the  upper  tail  of  the  equilibrium  distribution  in 
toward  the  mean  must  come  at  the  expense  of  pulling  the  lower  "tail"  in 
toward  the  mean  as  well,  /.a.  we  expect  some  sort  of  conservation  of  the 
mean  system  time  S.  In  a  single  work-conserving  G/G/1  queue  the  mean  is 
conserved  iff  packets  are  selected  independent  of  their  transmission  times 
[Kleinrock,  1976;  53.4]  [Heyman  et  a/,  1982;  511.5].  We  expect  the  situation 
to  be  a  little  more  complicated  in  the  network  case,  but  the  single  qu*ue  deri¬ 
vation  suggests  that  the  analogous  quantity  is  the  sum  of  transmission  times  on 
the  given  link  and  on  downstream  links.  Thus  if  we  give  priority  to  packets 
with  many  hops,  we  might  anticipate  an  increase  in  mean  delay  as  well  as  a 
reduction  in  higher  moments.  By  Little's  formula  [cf.  Stidham,  1972; 
pp.  1122-23]  this  would  increase  the  expected  number  in  system,  which  is  bad 
from  the  standpoint  of  congestion;  but  by  generalizations  of  Little's  formula 
[Brumelle]  [cf.  Kleinrock,  1975],  higher  moments  of  number  in  system  should 
be  reduced,  which  seems  desirable  from  a  congestion  standpoint. 

Insofar  as  the  mean  system  .  time  S  is  conserved,  we  can  alternatively 
view  convex  costs  as  being  generally  driven  by  the  central  moments 
/|t-S|kdF(t)  of  equilibrium  delay,  such  as  the  variance.  Indeed,  Jensen's 
inequality  [Ash;  p.  287]  gives  /c{t)dF(t)  i  c(S)  for  every  convex  function  c; 
moreover,  this  holds  with  equality  if  and  only  if  the  distribution  F  is  a  unit 


step  (i.e.  the  density  f  is  a  unit  impulse)  at  the  mean  S.  Thus  given  a 
collection  of  distributions  with  the  same  mean,  those  which  best  approximate 
(in  some  sense)  a  unit  step  are  the  most  desirable,  from  the  standpoint  of  any 
given  convex  cost;  this  is  essentially  a  restatement  of  the  basic  idea  of  $2.1. 

The  variance  of  equilibrium  delay  similarly  has  components  due  to  the 
variance  of  route  hop-length  and  the  variance  of  delay  at  individual  links  (or 
along  individual  routes).  Wong  et  a/,  have  recently  addressed  the  effect  of 
route  hop-length  variance  with  packet  scheduling:  they  derived  a 
time-dependent  scheduling  discipline  Ifif.  $3.3.2),  based  on  route  membership 
only,  that  minimizes  the  variance  I n(Sr-S)2G(r)  of  mean  route  delays  in  equilib¬ 
rium.  That  work  was  motivated  by  a  concern  for  fairness  among  different 
node  pairs  (cf.  $1.2),  and  the  scheme  effectively  aligns  the  means  of  the  densi¬ 
ties  fr(t),  subject  to  certain  connectivity  constraints.  However,  the  effect  on 
the  higher  moments  of,  delay  along  individual  routes  is  not  apparent.  We  also 
note  that  their  simulation  results  showed  little  variation  in  mean  delay  due  to 
scheduling  discipline. 

The  convexity  property  also  suggests  an  approach  to  lower  bounds  on  the 
expected  cost.  Denote  by  F(  ;*)  the  equilibrium  distribution  under  policy  »,  and 
define  S(»)  *  /tdF(t;»).  If  we  can  find  a  policy  »*  such  that  5(0  5  S(»)  for  all 
»cll,  then  for  every  convex  and  nondecreasing  function  c.  we  have 

c[S(0]  *  c[S<*)]  S  Jc(t)dF(t;i)  (2.20) 

by  monotonicity  and  Jensen's  inequality.  For  a  single  M/G/1  system,  it  is  well 
known  that  the  desired  »*  is  that  policy  which  gives  priority  at  each  departure 
instant  to  a  waiting  packet  with  the  smallest  transmission  time  (see  $3.2.2). 
From  the  discussion  above,  a  network  analogue  to  the  optimal  single  queue 
policy  should  suggest  itself.  However,  we  should  not  expect  such  bounds  to 
be  very  tight,  because  »*  is  not  likely  to  achieve  equality  in  the  second 


inequality.  Moreover,  the  idea  does  not  seem  to  generalize  to  the  case  of 
multiple  packet  classes  in  a  useful  way. 

Finally,  we  mention  the  possibility  of  improving  delay  performance 
through  selective  packet  routing.  There  are  a  great  many  approaches  to  rout¬ 
ing  [Tanenbaum;  $5.2],  but  the  effective  goal  can  be  expressed  as  follows: 
for  each  origin-destination  pair  of  nodes,  the  composite  traffic  flow  rate  (in 
bits/sec)  required  by  the  associated  users  is  apportioned  among  some  desig¬ 
nated  set  of  paths;  the  objective  is  a  profile  of  link  flows  that  gives  rise  to  a 
low  expected  path  delay  per  packet.  Given  such  s  flow  assignment,  then,  one 
could  reserve  the  lower  delay  paths  to  a  given  destination  for  the  more  urgent 
traffic.  However,  the  examples  given  in  $A.2  suggest  that,  for  larger  utiliza¬ 
tions,  such  a  scheme  will  not  in  general  be  as  flexible  as  scheduling. 


CHAPTER  III  -  SCHEDULING  ALGORITHMS  FOR  PACKET  NETWORKS 


Given  the  network  configuration,  the  packet  generation  process,  and  the 
delay  costs,  we  want  a  scheduling  policy  »  to  reduce  or  minimize  the 
performance  measure  J<»)  or  A  scheduling  policy  basically  specifies,  for 

each  departure  instant,  the  next  packet  to  be  transmitted.  An  urgent,  newly- 
arrived  packet  might  also  be  allowed  to  displace  or  "pre-empt"  the  one  in 
transmission.11  Without  allowing  pre-emption,  one  might  still  decide  to  leave 
the  transmitter  idle  after  a  departure  if  a  relatively  urgent  packet  were 
expected  in  a  very  short  while.  On  the  other  hand,  both  pre-emption  and 
inserted  idle  time  typically  complicate  queueing  system  analysis  as  well  as 
increase  the  expected  delay,  and  pre-emption  has  associated  protocol  problems 
in  packet  networks;  for  these  reasons  we  will  not  consider  such  strategies 
here. 

3.1  A  Decentralized  Scheduling  Problem 

The  network  scheduling  problem  arises  in  a  number  of  other  contexts,  for 
example  the  so-called  "job  shop"  scenario,  in  which  pieces  of  work  must  be 
processed  on  a  number  of  different  machines,  perhaps  not  in  the  same  order, 
[see  e.g.  Conway  et  a/].  In  many  such  cases,  the  time-scale  on  which  deci¬ 
sions  are  made  is  relatively  long;  it  is  then  reasonable  to  assume  that,  for  a 
given  realization  «,  the  characteristics  z„(«*)  «  <ar/»),bn(«),£h(«),r  n{«*)>  of  arrival  n 
(or  estimates  thereof)  become  known  to  a  single  scheduler  at  the  epoch  an(»). 

In  a  packet  network,  however,  this  information  becomes  available  at  an(«) 
only  at  the  first  link  in  the  route  rn(e).  Collecting  such  detailed  state  informa- 


xlFor  analysis  of  some  pre-emptive  queueing  policies  see  e.g.  Conway  et 
et.  [$8.7]  and  Schrage  [1968]. 
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tion  for  a  global  scheduler  (and  passing  instructions  back  to  the  individual 
nodes)  is  clearly  out  of  the  question;  at  the  same  time,  however,  most  of  this 
dynamic  information  is  of  relatively  little  value  in  making  scheduling  decisions 
for  a  given  link.  This  motivates  consideration  of  distributed  schemes,  in 
which  each  link  makes  its  own  scheduling  decisions,  based  on  a  combination 
of  locally  available  information  and  limited  communication  with  other 
nodes.11  A  reasonable  performance  measure  for  such  a  link  scheduler  would 
then  be  the  expected  (average)  cost  of  all  packets  transiting  that  link;  such  a 
scheduler  will  exert  a  minimal  influence  on  other  packets,  which  would  in  any 
case  be  difficult  to  quantify.  Thus  *  will  be  interpreted  as  a  link  scheduling 
policy,  to  be  executed  independently  at  every  link. 

Specifically,  we  will  consider  implementations  characterized  by  the 
following  assumptions  about  the  kinds  of  information  available  to  the  link 
scheduler: 

•  The  delay  cost  is  known  for  every  class;  we  do  not  antipicate  a 
large  number  of  classes  in  practice. 

•  Certain  sample  statistics  are  known  about  delays  on  the  routes  in 
which  the  link  is  included;  the  use  of  these  is  discussed  in  $3.2. 

Furthermore,  for  each  transiting  packet: 

•  The  epoch  of  arrival  to  the  link  itself  is  of  course  known. 

•  The  traffic  class  is  known. 

•  The  delay  at  upstream  links  in  the  route  (or  equivalently,  the  epoch 
of  arrival  to  the  system)  is  known;  this  has  obvious  practical 
implications,  which  are  discussed  below. 


iaSuch  an  approach  might  be  desirable  as  well  in  certain  applications  with 
longer  time  scales,  as  an  alternative  to  more  complicated  global  strategies. 


•  The  route  is  known;  actually,  we  do  not  need  the  route  per  se 
(which  may  not  be  known  in  practice),  just  some  key  to  the  delay 
statistics  associated  with  the  route,  such  as  the  session  identifier 
[Tanenbaum;  §§5.4.1,  5.4.4]. 


•  The  transmission  time  on  the  link  (or  equivalently  the  length)  is 
easily  established  during  arrival. 

Given  {zn}  and  *.  the  packet  process  seen  by  a  given  link  is  specified; 
however,  it  will  certainly  be  very  "messy,"  even  if  {ZrJ  is  "nice."  In  fact, 
the  link  arrival  process  will  in  general  have  a  mild  dependence  on  the  link 
departure  process,  and  hence  on  the  link  scheduling  policy,  through  feedback 
paths.  To  avoid  these  difficulties,  we  will  simply  define  a  new  link  process 
{y,j  from  scratch,  with  the  assumption  that  the  statistics  relevant  to  the 
scheduling  problem  are  similar  to  those  that  would  arise  unde  the  old  network 
process  {Zn}.  The  elements  of  yn  will  be:  the  epoch  an  of  arrival  to  the  link; 
the  class  b^  the  "history"  h,,  i.e.  the  time  behind  packet  n  in  the  system;  the 
route  tri  and  the  transmission  time  P  wilt  now  denote  the  probability 

measure  for  the  link  process.14  It  is  in  this  case  useful  to  make  the  explicit 
definitions 

y A")  *  <»n<‘*)*bn(»),hn((»),rn(e),Trt(**)>  f  IR*B*  [0,»)*R><(0,«>)  «  Y  (3.1) 

for  every  »«Q. 

The  assumption  of  accurate  knowledge  of  each  packets  history  is  an 
important  one  here.  For  example,  priority  schemes  based  on  number  of  hops 
in  route,  and  number  of  hops  remaining  in  route,  were  considered,  but  often 
had  much  worse  second  moment  properties  than  FCFS.  The  obvious  way  to 

14ln  principle,  knowledge  of  the  statistics  of  (yn)  might  help  the  link 
make  scheduling  decisions,  but  we  will  not  consider  this  approach  here. 

iSThus  no  additional  effort  is  entailed  by  a  round-trip-sensitive  reply 
packet  retaining  the  history  from  the  user  leg  of  its  trip  (cf.  §2.1). 


communicate  the  packet  history  is  for  every  packet  to  contain  a  bit  field, 
which  is  either  "stamped"  once  with  the  epoch  of  arrival  to  the  system,  or 
updated  by  each  node  in  the  route  to  reflect  the  new  history.1*  Thus  there  is 
an  obvious  tradeoff  between  resolution  and  link  loading,  and  the  assumption 
of  precise  information  can  never  really  be  satisfied.  On  the  other  hand,  we 
will  see  that  histories  ere  not  really  necessary  for  packets  with  linear  costs. 
In  other  cases  there  may  be  sneaky  ways  to  circumvent  the  requirement  as 
well:  for  instance,  in  a  packetized  voice  protocol,  a  precise  time  stamp  may 
be  forwarded  at  the  onset  of  every  session  (or  talkspurt,  as  the  case  may  be); 
the  system-arrival  epochs  of  subsequent  packets  can  then  be  recovered  from 
their  sequence  numbers  if  all  nodes  have  accurate  clocks.  However,  the 
relationship  between  resolution  and  performance  is  certainly  not  clear  and 
requires  further  attention. 

3.2  A  Dynamic  Programming  Formulation  of  Link  Scheduling 

The  behavior  of  a  queueing  system  may  often  be  modeled  as  a  contin¬ 
uous-time  Markov  process  [see  ».g.  Heyman  at  a/,  1982;  chs.  8,  9].  In  many 
cases,  however,  we  are  interested  in  the  system  behavior  only  at  a  certain 
(countable)  set  of  epochs,  and  the  behavior  on  this  "embedded"  set  of  epochs 
is  often  a  discrete-time  Markov  process  with  a  simpler  state  space  than  that 
of  the  underlying  process;  a  standard  example  is  the  M/G/1  queue  [Kleinrock, 
1975;  f5.3].  We  will  construct  such  a  Markov  rrcf.'i  for  the  link  scheduling 
problem,  or  more  precisely  a  model  "schema"  or  template.  The  model  is 
essentially  a  dynamic  programming  formulation  [Bertsekas,  1978;  chs.  2.  9];  it 
is  also  similar  to  the  Markov  decision  process  [Heyman  tt  #/,  1984;  ch.  4], 
For  other  applications  of  dynamic  programming  and  Markov  decision  processes 
to  queueing  system  control  see  ».g.  Stidham  #f  a/.  [1974]  and  references 


therein. 


3.2.1  Process  State  and  Dynamics 


We  first  need  a  notion  of  process  state.  We  want  to  make  a  scheduling 
decision  whenever  the  transmitter  is  idle  and  faces  a  non-empty  queue;  we 

will  represent  the  kth  such  decision  epoch  by  tk.  k  ■  0, 1 . 

We  want  also  to  represent  the  characteristics  of  the  packets  in  queue. 
Let  Q  be  a  subset  of  IN*,  and  •  a  function  from  Q  into  Y;  the  function  »  can 
be  interpreted  simply  as  a  list  of  packets  and  their  characteristics;  indeed,  the 
rigorous  definition  of  a  function  »:Q-Y  is  the  set  of  ordered  peirs  {<n,y>:  ncQ, 
y*»(n)}  [see  e.g.  Suppes;  ch.  3].  Then  if  Y  is  the  set  of  all  such  lists,1*  we 
may  represent  by  »k:Q-»Y  the  queue  list,  and  by  Qk  the  set  of  packets  in 
queue,  at  the  decision  epoch  t* 

Finally,  in  order  that  the  process  be  Markov,  we  typically  need  to  record 
some  additional  information  about  the  process  history  up  to  epoch  tk.  We 
denote  this  information  by  jtk:0«>*,  where  V  is  an  appropriate  space;  the  nature 
of  the  f k  will  be  elaborated  on  below.  Thus  we  define  the  state  represen¬ 
tation  xk  at  the  kth  decision  epoch,  and  the  state  space  X,  by 

xk(«)  a  <tk(«).*k(w),?k{«)>  *  IR*Y*T  ■  X;  (3.2) 

note  that,  given  »«Y,  the  corresponding  packet-set  Q  is  always  determined  as 
(n:  <n,y>«»  for  some  y*Y},  i.e.  as  the  domain  of  ».  In  particular,  the  initial 
state  x0  will  include  components  defined  by 


to  *  •  i»  (3.3) 

Q0  ■  [n:  an-a,}.  (3.4) 


14Note  that  every  such  function  »:Q-Y  is  a  subset  of  IN**Y  *  {<n,y>: 
n«IN\  yrY}  (think  of  the  graph  of  *  in  IN*-Y  coordinates);  then  Y  is  the  space 
of  all  subsets  of  IN*«Y  which  are  also  functions.  Other  "big"  sets  defined 
here  (such  as  II)  can  be  precisely  specified  in  a  similar  manner. 
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*  »o  *  {<n,yn>:  neQ0}.  (3.5) 

We  now  represent  the  dynamics  which  govern  the  motion  of  the  process 
through  the  state  space.  We  will  say  a  map  /i:X-IN+  is  admissible  if  /i(x)<Q 
for  all  x«X.  Then  we  formally  define  (at  last)  an  admissible  scheduling  policy 
as  a  sequence  »  •  <p0.pi....>  of  admissible  maps  on  the  state  space;  let  n  be 
the  set  of  all  such  policies.  The  interpretation  is  that,  at  the  epoch  tk,  the 
scheduler  selects  for  transmission  the  packet 

uk  ■  /«k<xk)  <  Qk  (3.6) 

I  (in  the  finite  case  only  the  first  N  maps  are  used);  packet  uk  subsequently 

departs  at  epoch  d(uk)  ■  tk  ♦  r(uj.  A  policy  fell  is  stationary  if  *  ■  </i,  /»,...> 
for  some  /». 

j  The  next  decision  epoch  can  be  written  as 

ak+1  if  Qk-uk  &  ak4 1  >  d(uk), 
tk*i  ■  {  (3.7) 

d(uk)  otherwise 

I  where  a  is  the  empty  set.  Let  Ek<M  ■  {n:  tk<anStk*1}  be  the  set  of  packets 

which  enter  the  queue  in  the  interval  following  the  kth  decision;  then 

'  Qk+i  *  Qk  -  uk  U  Ek,  (3.8) 

and  the  new  queue  list  is  given  by 

!  *k+i  *  {<n.yn>:  n€Qk*,}.  (3.9) 

I 

For  a  given  policy  *,  denote  by  (xk(v)}  the  process  which  results  from  the 
sequence  of  decisions  (uk*/ik(xk),  kSO). 

As  a  concrete  example  of  how  the  fk  would  be  chosen,  suppose  that 

I  * 

{y is  the  superposition  of  independent  processes  for  each  bcB,  and  that  for 


.V.  -•  ,V 


j  each  component  process,  the  interarrival  times,  histories,  routes,  and  transmis¬ 

sion  times  are  i.i.d.  and  independent  of  each  other;  then  it  is  suffucient  that  fk 
includes  for  every  class  the  latest  arrival  epoch  up  to  and  including  tk.  On 
|  the  other  hand,  a  model  of  this  type  may  in  principle  be  constructed  for  the 

system  behavior  under  completely  arbitrary  packet  processes,  by  recording  the 
entire  state  history  in  the  pk. 

I 

3.2.2  Decision  Costs 

The  process  {y^  says  nothing  about  the  probabilistic  behavior  of  packets 
after  they  leave  the  link.  As  the  basis  of  a  performance  measure  for  the  link 
scheduling  problem,  we  will  use  the  delay  statistics  (and  the  delay  costs)  to 
develop  a  decision  cost  g:  IR*  Y  -  [0,«);  here  g(t,y)  will  represent  an  estimate  of 
the  cost  (on  system  exit)  for  a  packet  with  characteristics  y  which  begins 
|  transmission  at  epoch  t  on  a  given  link.  Thus  g[tk.y(uk)]  will  represent  the 

estimated  cost  for  the  packet  uk  selected  at  epoch  tk,  and  we  may  redefine 
JN  and  J  as  (expected)  estimated  costs  of  using  the  policy  »  by11 

I 

JnI.)  *  IE  la  [t.l.l.ylu.)]),  (3.10! 

k«0 

J(»)  ■  lim  <i/N)NZ  E{g[tk(»).y(uk)]}.  (3.11) 

Nt  k  “0 

where  y(u,J  abbreviates  y{/»k[xk(*)3 }.  It  is  not  hard  to  show  that  (2.8)  and 
(3.10)  are  equivalent,  in  the  sense  that  they  would  have  the  same  value  in  a 
single  link  system;  assuming  convergence,  we  might  expect  the  same  of  (2.9) 


‘’It  is  typical  in  dynamic  programming  to  condition  the  performance 
measure  on  a  given  initial  state;  in  the  present  context  we  expect  any  such 
dependence  to  be  very  mild  at  worst  and  we  will  not  worry  about  it  explicitly. 


-  .> 
'.Tv*  O 
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and  (3.11),  although  a  proof  it  not  attempted  here.  The  link  scheduling  prob¬ 
lem  is  then  effectively  specified  by  a  probability  measure  and  decision  cost 
<P,g>;  given  such  an  instance  of  the  problem,  a  policy  which  minimizes  (3.10) 
or  (3.11)  will  be  called  optimal. 

Our  approach  to  the  decision  cost  g  at  a  given  link  is  to  construct  an 
estimate  f(t,y)  of  the  "future,"  /.*.  of  the  delay  on  downstream  links,  for  a 
packet  with  characteristics  y  selected  at  t;  we  then  define 

g(t,y)  *  c[b,  h  ♦  (t-a)  ♦  r  +  f].  (3.12) 

Of  course  the  actual  packet  futures  must  have  some  variation  about  the  esti¬ 
mate  f.  and  for  strictly  convex  delay  costs  this  will  drive  up  the  average  cost 
beyond  the  estimate  given  by  (3.12).  In  certain  such  cases,  the  bias  is  irrel¬ 
evant  to  the  optimization  problem:  for  instance,  if  c(b,t)  «  ta,  then  g(t.y)  * 

(h  +  t-a«-  r  +  f)2  ■  ta  ♦  2(h-a*r+f)t  +  (h -a  +  r  ♦  f)a,  and  the  fa  term  is  an 
isolated  constant.  For  a  cubic  delay  cost,  however,  the  slope  of  the  decision 
cost  would  be  biased  by  an  f 2  term  (in  this  case  one  might  consider  the  addi¬ 
tional  use  of  an  estimate  of  the  second  moment  of  downstream  delay).  As 

the  convexity  becomes  more  pronounced,  however,  we  should  also  expect  that 

an  effective  scheduling  algorithm  will  tend  to  reduce  the  variation  in  packet 
futures  for  any  given  <t,y>. 

Thus  the  link  essentially  interprets  the  downstream  estimate  f  as  a  delay 
which  has  already  been  incurred;  it  is  then  not  completely  unreasonable  to 
define  the  effective  elapsed  time 

y(t.y)  ■  h  +  (t-a)  ♦  f(t,y)  (3.13) 

of  a  packet  with  characteristics  y  at  epoch  t.  The  link  is  thus  assumed  to  be 
already  "liable"  for  the  amount  c[b,y(t,y)]  at  epoch  t,  and  the  decision  cost 


may  now  be  expressed  simply  in  terms  of  the  effective  elapsed  time  as 
gtt.y)  ■  c(b,i+r). 

To  conatruct  our  future  estimate  f,  we  assume  that  the  link  scheduler  has 
a  table  which  gives  an  estimate  D(b,r)  of  the  delay  to  the  destination  on 
downstream  links  for  a  class-b  packet  on  route  r;  of  course  D(b.r)  »  0  if  the 
link  is  the  last  one  on  route  r.  In  practice,  the  estimates  are  likely  to  be 
sample  moving  averages  of  the  form 

(M-m)*1  "l*  Di(b,r, #,*),  (3.14) 

J 

where  Dj(b,r,«,*}  is  the  downstream  delay  of  the  jth  class-b  packet  on  route  r 
to  reach  the  destination.  The  table  would  be  updated  periodically  using 
reports  from  the  destination  nodes.  Adaptive  routing  algorithms  often  employ 
such  tables  of  downstream  delays  (for  a  single  class)  [Tanenbaum;  $5.2].  For 
simplicity,  we  will  assume  that  0  is  independent  of  m  and  »,  since  we  have  no 
intention  of  accounting  for  these  complications.  Thus  we  obtain  a  simple 
estimate  of  the  future  by  setting1* 

f(t.y)  «  D(b.r).  (3.15) 

Note  that  f  is  independent  of  t  here,  so  that  g(  ,y)  is  simply  a  shifted  version 
of  c(b,  ),  and  has  the  same  functional  form. 


“And  this  can  similarly  be  modified  to  include  an  estimate  of  the  system 
time  on  the  return  leg  for  a  round-trip-sensitive  user  packet. 


3.2.3  A  Refinement  of  the  Downstream  Delay  Estimate 

For  a  given  pair  <b,r>,  the  goal  of  the  scheduler  may  be  viewed  as  mini¬ 
mizing  the  variation  in  estimated  system  times  of  transiting  packets  (cf. 
$2.3.2).  The  packet  histories,  arrival  epochs,  and  transmission  times  will  be 
taken  into  account  by  the  downstream  links,  and  this  should  help  rectify 
imbalances  in  the  delays  accumulated  at  the  given  link  and  the  upstream  links 
associated  with  transiting  routes;  the  estimate  f  ■  D(b,r)  is  somewhat  "pessi¬ 
mistic"  in  that  is  does  not  anticipate  any  such  help  from  the  downstream 
links.  The  following  example  illustrates  how  this  may  degrade  system 
performance. 

Suppose  for  a  given  realization  there  are  four  packets  in  queue,  all  of  the 
same  class  b  and  with  identical  transmission  times  r.  The  link  is  the  last  one 
in  a  route  shared  by  packets  1  and  2,  so  their  estimate  is  f  1,2  *  0;  their 
accumulated  delays  differ  by  3r  seconds.  The  link  is  the  first  one  in  a  route 
shared  by  packets  3  and  4,  with  some  estimate  fa,4;  they  have  arrived  5r 
seconds  apart. 

This  is  clearly  the  last  opportunity  to  do  anything  about  the  relative 
delays  of  packets  1  and  2,  and  in  fact  their  system  times  can  be  made  exact¬ 
ly  equal  if  the  schedule  1-3-4-2  is  followed.  This  schedule  also  reduces  by  r 
the  difference  in  accumuleted  delays  for  packets  3  and  4  (as  seen  by  the  next 
link)  to  4r;  moreover,  there  is  a  good  chance  that  this  difference  can  be 
reduced  further,  if  not  eliminated  entirely,  by  the  downstream  links.  But  the 
pessimistic  scheduler  is  in  effect  operating  under  the  mistaken  assumption  that 
this  is  also  the  last  opportunity  to  do  anything  about  packets  3  and  4.  Fig.  4 
shows  how  the  effective  elapsed  times  of  the  four  packets  might  be 
related  in  the  worst  case;  the  average  decision  cost  at  the  link  is  then  mini¬ 
mized  by  the  schedule  3- 1-2-4  (this  is  easy  to  prove,  and  we  do  so  in  $3.4, 
but  the  intuition  should  be  fairly  clear).  This  schedule  indeed  reduces  the 


differences  in  accumulated  delays  for  packets  3  and  4  to  2r,  but  results  as 
well  in  an  irrevocable  difference  of  2r  in  the  total  system  times  of  packets  1 
and  2. 


We  can  also  imagine  the  opposite  extreme,  that  of  the  totally 

"optimistic"  scheduler;  this  one  assumes  that  the  downstream  links  are  some¬ 
how  able  to  guarantee  equal  system  times  for  all  packets  with  the  same  <b,r> 
(a  particularly  poor  assumption  on  the  last  link  in  a  route).  To  generate  such 
optimistic  estimates  of  downstream  delays,  a  fink  might  maintain  a  table 
which  gives  an  estimate  S(b,r)  of  the  total  system  time  for  a  class-b  packet 
on  route  r,  and  derive  a  time-dependent  estimate  f(t,y)  «  S(b.r)  - 

[h  ♦  (t-a)  +  r].  This  gives  *(t,y)  ■  S(b,r)  -  r  and  g(t.y)  «  c[b,S(b,r)];  with  this 
estimate,  then,  decisions  may  be  based  on  at  most  packet  class  and  route,  and 
cannot  be  very  meaningful. 

The  example  of  Fig.  4  shows  that  a  link  has  a  limited  capacity  for  reduc¬ 
ing  variations  in  accumulated  delay.  Thus  a  more  "realistic”  scheduler  should 
expect  that,  for  a  given  pair  <b,r>,  each  downstream  link  will  make  some 
contribution  to  this  goal,  and  it  should  realize  that  some  effort  on  its  own 


Figure  4  Elapsed  times  with  pessimistic  estimates 
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part  will  be  required  as  well.  We  may  strike  such  a  balance  between  the 
pessimistic  and  optimistic  schedulers  with  an  estimate  of  the  form 

f(t,y)  «  ^yD(b.r)  -  (1-Jy)  [S(b,r)  -  (h  ♦  t  -  a  ♦  r)]  (3.10) 

where  ly<(0,1)  represents  the  fraction  of  the  responsibility  assumed  by  the  link 
for  a  packet  with  characteristics  y. 

As  one  possibility,  if  jr  is  the  number  of  downstream  links  on  route  r, 
we  might  set  *  l/(jr+l).  Then  for  packets  at  early  links  in  a  route,  the 
differences  in  effective  elapsed  times  are  reduced  to  much  more  reasonable 
values  than  those  seen  by  the  pessimistic  scheduler;  however,  they  are  not 
completely  eliminated,  so  that  meaningful  scheduling  decisions  may  still  be 
made.  Of  course  other  assignments  for  are  possible,  and  experimentation 
would  probably  be  helpful  in  establishing  reasonable  choices;  but  one  obvious 
constraint  is  that,  on  the  last  linfc  in  a  route,  should  go  to  1,  so  that  f 
goes  to  0. 

Now.  however,  f  is  affine  in  t,  and  we  find  that 

g(t.y)  «  c(b.^yt  +  Ky)  (3.17) 

for  some  Ky  independent  of  t.  While  the  convexity  of  c(b,  )  is  retained  in 
g(  ,y),  the  particular  functional  form  need  not  be;  we  will  see  in  $3.4  that  the 
characterization  of  an  optimal  policy  becomes  more  difficult  in  this  case. 

3.2.4  Performance  on  a  Saturated  Link 

Consideration  of  scheduling  disciplines  other  than  FCPS  opens  up  the 
possibility  that  some  packets  may  get  "trapped"  in  the  system  forever,  and 
we  clearly  want  to  avoid  this.  Indeed,  both  (2.9)  and  (3.11)  have  little  signi¬ 
ficance  unless  (with  probability  one)  sn  is  finite  for  every  n.  If  this  condition 


is  not  met,  then1*  E  [efb^Sn)]  *  *  for  some  n,  so  that  (2.9)  cannot  converge. 
And  because  (3.11)  accounts  for  packets  in  order  of  departure,  it  doesn't  even 
give  any  indication  of  this  problem;  in  fact  the  scheduler  might  conceivably 
minimize  (3.11)  while  neglecting  indefinitely  the  most  urgent  packets. 

To  ensure  no  trapping,  we  could  require  that  on  each  link  the  queue 
empties  out  infinitely  often,  i.e.  the  queue  busy-periods  are  all  finite  [this  is 
typically  sufficient  for  stability  of  a  queue:  Kleinrock,  1975;  p.  93];  then  every 
packet  is  selcted  eventually,  at  every  link  in  its  route.  More  precisely, 
suppose  that,  on  every  link,  we  have 

(w.p.1)  liminf  |Qk|  *  1,  (3.18) 

where  |Q|  denotes  the  number  of  elements  in  a  set  Q;  recall  that  by  defi¬ 
nition  1 0K |  is  never  0.  Thus  for  every  k  there  exists  a  j>k  such  that 
|  Qj  |  •  1  [cf.  Rudin;  pp.  55-57];  then  <w.p.1)  for  every  packet  n,  we  have  n«Ek 
for  some**  k.  and  hence  dn  S  tj  ♦  rn  and  sn  i  tj  -  tk  +  rn  for  some  j>k. 

We  might  also  imagine  a  situation  in  which  the  finite  busy-period 
assumption  is  violated,  particularly  when  the  potential  traffic  is  large.  As  a 
somewhat  weaker  assumption,  then,  we  require  only  that 

(w.p.1)  linr^inf  |Qk|  ■  M  <  •  (3.19) 

for  some  M,  /.a.  for  every  k  there  exists  a  j>k  such  that  |Qj|  s  M;  we  do 
expect  thet  such  an  M  exists  in  practice,  regardless  of  the  policy.  We  may 


**  Suppose  that,  for  every  n,  E[c(bn,Sn)]  is  finite:  then  by  Jensen's  inequal¬ 
ity  E(sn)  is  finite,  hence  sn  is  finite  w.p.1;  then  the  probability  o<  *e  set  of 
reelizations  in  which  sn  is  infinite  for  some  n  is  P(Un2i{s>n  «*rinite})  S 
ZnziPUn  infinite]  *  0. 


’•this  actually  presumes  in  addition  that  (w.p.1)  liminfnIrn  >  0. 


loosely  refer  to  e  link  for  which  M>1  as  marginally  stable  (pathological  insta¬ 
bilities  exist).  Trapping  is  now  possible  for  some  policies,  which  suggests 
(correctly)  that  (3.18)  is  also  a  necessary  condition  (for  no  trapping  under  every 
policy);  but  an  argument  similar  to  the  one  in  the  previous  paragraph  shows 
that  no  trapping  occurs  under  FCFS,  so  that  at  least  one  such  policy  always 
exists. 

We  will  address  the  marginally  stable  case  by  holding  the  link  account¬ 
able  for  every  arriving  packet,  as  follows:  in  $3.3.1,  r*(t,r)  is  defined  as  the 
optimal  cost  of  emptying  out  the  queue  described  by  the  list  #,  starting  at 
epoch  t;  we  redefine  J  (for  the  last  time)  by*1 

J(»)  *  lim  (1/N)  E{NZg[tk(,).y(uk)]  ♦  P  [t  „(*),.„{,)]}.  (3.20) 

N  l  k-0 

r*(tffr0N)  represents  the  liability  of  the  link  after  N  decisions  (assuming  no 
additional  arrivals).  Now  if  (1/N)E{r*[t„<(*),»N()r)]}  -  0,  then  obviously  J(»)  * 
limNt(1/N)Jn(v).  Moreover,  we  expect  intuitively  that  (w.p.1)  no  trapping  occurs 
in  this  case,  and  that  (2.9)  and  (3.20)  are  equivalent. 

It  would  be  nice  if  we  could  guarantee  that  (w.p.1)  the  optimal  policy 
never  traps  in  the  marginally  stable  case,  but  this  is  not  true;  for  instance,  in 
the  case  of  linear  delay  costs  c(b,t)  *  mbt  and  constant  transmission  times,  we 
will  show  in  $3.3.2  that  (in  the  single-queue  case)  packets  with  minimum  mb 
always  have  the  lowest  priority,  regardless  of  their  effective  elapsed  times. 
In  principle,  there  should  exist  conditions  under  which  the  optimal  policy  will 
never  trap;  a  likely  candidate  would  be  unbounded  marginal  costs  for  every 
class.  In  such  e  case,  T*  would  serve  as  a  "penalty  function"  which  discour- 

*lThus  J  is  no  longer  a  true  per-packet  average;  this  allows  for  further 
anomalies,  which  we  will  not  pursue,  but  the  reader  is  invited  to  explore. 


ages  the  scheduler  from  neglecting  any  particular  packet  for  too  long.  In 
practice,  however,  we  cannot  guarantee  such  a  condition.  Alternatively,  we 
could  establish  absolute  time  limits  such  that  a  packet  automatically  goes  onto 
a  special  subqueue  with  preferred  status  once  its  limit  has  been  reached;  this 
will  also  guarantee  no  trapping. 

3.3  A  Class  of  Efficient  Scheduling  Policies 

It  is  appropriate  in  this  context  to  focus  on  simple  policies  which  can  be 
executed  quickly.  The  transmission  time  of  a  data  packet  is  typically  on  the 
order  of  10  milliseconds;  thus  even  an  (inexpensive)  processor,  dedicated  to 
the  task  of  scheduling,  will  have  time  for  only  a  few  thousand  machine 
instructions  per  decision. 

3.3.1  Myopic  Scheduling  Policies 

In  particular,  it  seems  reasonable  first  of  all  to  forget  about  the  dynamic 
aspect  of  the  problem;  i.e.  at  any  given  decision  epoch,  ignore  the  fact  that 
more  packets  are  expected  to  arrive  in  the  future,  and  address  only  the  static 
problem  of  emptying  out  the  queue  at  minimum  cost.  Policies  of  this  type 
will  be  called  myopic  scheduling  policies  [cf.  Heyman  et  at,  1984;  ch.  3];  the 
notion  will  not  really  demand  a  precise  definition. 

In  general,  the  arrival  of  new  packets  can  upset  the  optimality  of  a  static 
schedule,  as  the  following  example  shows.  Consider  a  single  link  system,  and 
suppose  that  for  a  given  realization  packets  1  and  2  are  present,  and  packet  3 
will  arrive  in  r  seconds.  All  packets  have  transmission  times  r.  The  delay 
costs  are  c(b  vt)  *  £max{0,t-r},  c(b»t)  ■  «max{0,t-2r},  and  c(b»t)  a 
«max{0,t-r},  with  c  «  a,  as  shown  in  Fig.  5.  The  optimal  sequence  for  pack¬ 
ets  1  and  2  is  clearly  1-2,  which  has  total  cost  0.  But  if  packet  1  goes  first, 
then  one  of  packets  2  or  3  must  incur  a  cost  of  «r,  whereas  the  policy  which 


results  in  2-3-1  has  total  cost  2 tr  *  ar;  depending  on  the  process  statistics, 
then,  the  myopic  policy  need  not  be  optimal,  in  certain  cases,  however,  a 
myopic  policy  can  minimize  the  expected  cost**  as  defined  in  $3.2. 

To  formalize  the  static  scheduling  problem,  let  **T  be  a  queue  list  with 
|  Q  |  ■  q  packets.  We  define  a  static  sequence  for  •  as  a  collection 
a  •  {*„  ncQ},  where  *n  is  the  set  of  packets  in  Q  to  precede  packet  n.  The 
an  must  then  be  strictly  nested,  i.e .** 

0  •  *(n  i)  <  .  .  .  <  *(ncj  ■  Q-n^  (3.21] 

where  n,  is  the  first  packet  according  to  the  sequence  a  end  nq  is  last.  Given 
a  decision  cost  g.  the  cost  of  scheduling  »  in  accordance  with  a,  starting  at 
epoch  t,  may  then  be  written  compactly  as 

I>.t.*)  ■  I  g[t  *  I  rj,  #(n)],  (3.22) 


Figure  5  How  myopic  policies  can  fail 


**There  are  other  such  sequential  decision  problems,  e.g.  some  inventory 
problems,  for  which  the  optimal  policies  are  myopic  under  certain  conditions 
[see  Heyman  et  el,  1984;  ch.  3]. 


**Alas,  "<"  denotes  proper  set  inclusion. 


and  this  motivates  our  definition;  but  we  will  often  revert  to  the  shorthand 
n^j-^.-nq.  The  optimal  cost  for  the  static  problem  <t,»>  will  be  defined  as 

r‘(t.*)  ■  min  n».t,*l  (3.23) 

a 

(cf.  $3.2.2),  and  the  sequence  a*  is  optimal  for  the  problem  <t,#>  if  it  achieves 
the  minimum  in  (3.23). 

There  are  q!  possible  static  sequences  for  a  given  collection  of  q 
packets;  the  search  for  more  efficient  solutions  to  this  problem  has  been  rela¬ 
tively  active  over  the  last  twenty  years.  There  seem  to  be  two  basic 
approaches,  the  first  of  which  consists  of  devising  rules  for  eliminating 
non-optimal  sequences  from  consideration.  As  the  most  straightforward  exam¬ 
ple,  dynamic  programming  approaches  can  reduce  the  computation  required  in 
the  general  problem  to  the  order  of  2q  [Held  et  a/.]  [Schild  et  a/.]  [Lawler, 
1964],  When  the  cost  functions  take  a  particular  form,  we  can  apparently  do 
better;  additional  elimination  criteria  have  been  given  for  non-decreasing  costs 
[Elmaghraby] ,  CND  costs  [Henderson  at  a/;  $3],  and  quadratic  costs  [Schild  et 
a/].  But  there  is  nothing  to  suggest  that  the  complexity  of  these  schemes  is 
better  than  exponential,  which  is  unacceptably  slow  in  our  context.  Moreover, 
it  is  not  clear  that  it  is  any  less  difficult  to  find  the  first  item  in  an  optimal 
sequence  (which  is  all  we  really  want  for  a  dynamic  policy)  than  it  is  to  find 
the  whole  sequence. 


3.3.2  Priority  Rules 

The  second  epproech  is  the  use  of  intuition  (end  a  little  luck)  to  find 
selection  rules  for  the  first  item  when  the  decision  costs  have  e  particularly 
auspicious  form;  these  rules  can  typically  be  expressed  as  priority  rules,  which 
are  (at  worst)  order  m  at  each  decision,  and  this  is  as  good  as  we  could 
reasonably  have  hoped  for.  We  may  interpret  any  function  f:  IR*Y  -  IR  as  a 
priority  function;  when  a  decision  is  to  be  made  among  the  packets  in  a  queue 
list  »  at  epoch  t,  we  simply  select  3  packet  ncQ  for  which  f  [t,#(n)]  is  maxi¬ 
mum.  i.e.  we  select  a  packet  with  the  highest  priority  [cf.  Schrage,  1973];  for 
concreteness  (and  without  loss  of  generality),  we  assume  that  ties  are  resolved 
in  favor  of  the  lowest  index.  If  ^(t.y)  is  independent  of  t  for  all  yeY,  then  f 
is  time- independent,  and  we  may  as  well  write  ^:Y-»IR;  most  useful  priority 
rules  are  in  fact  of  this  type.  Time-independent  rules  are  especially  attractive 
here  because  packets  need  only  be  ranked  once  [e.g.  inserted  into  a  linked-list 
data  structure);  this  of  course  represents  less  overall  computation,  but  also 
facilitates  consideration  of  packets  arriving  relatively  close  to  the  decision 
epoch. 

Given  e  static  problem  <t,  *>  we  define  the  sequence  o  generated  by  a 
time-independent  priority  function  f  (or  the  corresponding  rule)  by 

•  {icQ:  f[w{ i)]  >  *[.(])]}  U  {i«Q:  *[r(i)]  -  f  [*{j>]  &  i<j)  (3.24) 

for  all  jcQ;  the  time-dependent  case  is  tedious  to  write  down,  but  should  be 
intuitively  clear  as  well.  Given  a  decision  cost  g,  we  will  say  that  a 
time-independent  priority  function  (or  rule)  f*  is  optimal  (in  the  static  sense)  if 
it  generates  an  optimal  sequence  for  every  <t,r>  t  IR*  Y.  For  certain  decision 
costs,  optimal  time-independent  rules  do  exist  (we  are  aware  of  no 
time-dependent  rules  which  are  optimal  in  this  sense).  Similarly,  the  (station¬ 
ary)  policy  t  generated  by  a  priority  function  f  (or  rule)  may  be  defined  by 
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idt.v)  *  min{ieQ:  f  [t,*(i)]  i  f[t.»(j)]  for  all  j«Q) 


(3.25) 


for  all  xcX.  where  the  notation  emphasizes  the  fact  that  decisions  depend  only 
on  t  and  v,  similarly,  for  a  time-independent  rule  we  write  /*(»).  The  policy 
generated  by  an  optimal  priority  rule  may  be  optimal  as  well  for  certain 
statistics  P. 

To  illustrate  these  ideas,  priority  functions  corresponding  to  a  number  of 
well-known  selection  rules  (usually  associated  with  single-queue  systems)  are 
now  presented.  First-come-first-served  may  be  expressed  as  f(y)  >  -a.  In  the 
case  of  linear  decision  costs  g(t.y)  *  myt,  the  so-called  "pC"  rule  is  given  by 
^(y)  *  m y/r;  it  is  optimal  for  the  static  problem,  as  we  will  demonstrate 
below,  and  the  policy  it  generates  is  optimal  for  the  dynamic  problem  (in  the 
stable  case)  when  the  arrival  process  is  Poisson  and  the  classes  and  service 
times  are  i.i.d.  and  independent  [Lippman]  [see  also  Fife].  In  the  special  case 
g(t,y)  «  t,  the  "shortest  job  first"  rule  mentioned  in  §2.3  is  of  course  given  by 
^(y)  «1/r.  Now  consider  costs  of  the  form  c(b,t)  *  maxlO.t-Tfc};  Tb  has  the 
interpretation  of  a  due-date  or  grace  period,  after  which  a  linear  cost  is 
incurred.  Jackson's  "due-date  rule"  is  given  by  f(y)  *  -(Tb-i),  i.e.  select  the 
packet  whose  due-date  is  closest;  for  a  single  queue,  the  rule  can  be 
expressed  as  f[ y)  «  -(Tb*a),  and  minimizes  the  "maximum  lateness" 
maxNldbrySn)}  in  the  static  problem  [Conway  et  a/;  §§3.3,  8.8].  Finally, 
Kleinrock's  time-dependent  rule  [1976;  §3.7]  is  the  one  used  by  Wong  et  at.  to 
place  the  means  of  the  various  route  delays  (see  §2.3.2  above);  it  is  probably 
most  meaningfully  expressed  here  as  ^(t,y)  *  mr(t-a),  where  the  mr  are  suit¬ 
able  constants,  chosen  independently  for  every  link. 

It  is  easy  to  show  that,  in  the  case  of  linear  decision  costs  g(t,y)  »  myt, 
the  rule 


f  *(y)  «  m  y/r 


is  optimal  (in  the  static  sense).*4  Let  <t,t>  be  given,  and  let  r*  be  the 
sequence  generated  by  }*;  without  loss  of  generality,  assume  that  Q  > 

(1,2, _ q)  and  **(1)  <  <r*(2)  <  ...  <  **(q);  then  n#’.t,»)  5  T{e,t,»)  for  every  #, 

as  follows.  If  packet  1  is  not  first  in  a,  we  can  interchange  it  with  the 
immediately  proceeding  packet  j  without  increasing  the  total  cost:  the  costs 
of  the  other  packets  clearly  are  unaffected  by  the  interchange  (because 
ri  +  rj  *  tj  +  r,);  the  cost  of  packet  1  decreases  by  m^j  and  the  cost  of 
packet  2  increases  by  m^rv  and  m1/r1  i  m  j/r  j  yields  for  the  net  cost 
increase  mjr,  -  m  ,r  s  0.  Iterating  the  argument,  packet  1  can  be  "bubbled" 
to  the  first  position  in  the  sequence  without  increasing  the  cost,  and  packets 
2, 3,...q-1  can  similarly  be  bubbled  to  the  2nd, 3rd,...(q-l)st  positions  without 
increasing  the  cost.  In  general,  the  optimality  of  any  time-independent  rule 
may  in  principle  be  established  through  such  an  interchange  argument,  because 
the  rule  will  give  an  optimal  sequence  for  the  two-packet  problem  correspond¬ 
ing  to  any  interchange  pair. 

It  might  seem,  at  first  glance,  that  an  optimal  priority  rule  should  always 
generate  an  optimal  policy,  but  this  is  not  so.  The  following  example  is  a 
little  involved,  but  illustrates  some  pertinent  points.  Consider  a  single  queue 
with  linear  costs  c(b,t)  *  mbt,  using  the  /»C  rule  f{y)  •  m Jr.  Suppose  that,  for 
a  given  realization,  packets  1.  2,  and  3  have  the  following  transmission  times, 
delay  costs,  and  priority  assignments: 


ri  ■  r 

• 

c(bvt)  «  t 

fiy  i>  ■  1  It 

t  2  *  (1  ♦  2e)r 

c(b2,t)  ■  (1  «■  e)t 

f( y2)  *  (1  4  e)/(1  +2c)r 

r3  •  r 

c(ba,t)  ■  2t 

^(y3)  «  2 It 

* 4 Although  the  proof  is  simple,  it  is  new  to  us.  Proofs  of  optimality  in 
the  dynamic  case  proceed  differently. 


where  c  «  1.  Peckets  1  and  2  arrive  simultaneously,  and  packet  3  arrives 
(1  *2e)r  seconds  later.  Then  the  policy  generated  by  the  pC  rule  (which  is 
indeed  optimal  for  packets  1  and  2  only)  results  in  the  sequence  1-2*3,  with 
total  cost  (7  +  4«  +  2«a)r;  on  the  other  hand,  the  sequence  2-3-1  has  total  cost 
(6  ♦5*  *2t2)r. 

It  is  fairly  clear  that  the  scheduler  is  forced  into  making  a  poor  decision 
because  of  the  variation  in  transmission  times  (the  example  suggests  as  well 
the  possible  benefits  of  inserted  idle  time);  indeed,  if  we  reset  rn  «  r  for  all 
n,  the  fiC  rule  reduces  to  a  "maximum  slope"  rule  which  gives  the  optimal 
sequence  2-1-3;  similarly,  if  rn  •  (1  +  2*)r  for  all  n,  the  rule  correctly  gives 
2-3-1.  It  turns  out  that,  in  the  case  of  constant  transmission  times,  every 
optimal  time-independent  rule  always  </.e.  for  every  P)  generates  an  optimal 
policy  (in  fact,  it  minimizes  the  cost  for  every  realization);  the  proof  is  based 
on  interchanges  of  non-adjacent  packets,  and  is  given  in  $A.3.  Thus  in  the 
case  of  linear  decision  costs,  the  maximum  slope  rule  generates  an  optimal 
policy  for  the  G/D/1  link. 

3.4  Selection  Rules  for  Strictly  Convex  Costs 

The  /iC  rule  may  in  some  cases  be  a  useful  guideline  even  when  the 
delay  costs  are  not  linear.  Given  a  problem  <t,v>  in  the  single-queue  case,  if 
the  marginal  costs  m(b,  )  can  be  reasonably  approximated  as  constant  over  the 
corresponding  intervals  [y,  »  +  IorrO'  then  the  interchange  argument  of  $3.3.2 
has  some  validity,  and  the  use  of  the  priority  rule 

f{ t.y)  »  m  [b,  f(t.y)]  It  (3.27) 

** Moreover,  there  probably  exist  coupled  constraints  on  the  ranges  of  the 
marginal  costs  and  transmission  times  under  which  an  optimal  sequence  is 
guaranteed. 


should  result  in  good  performance 1  *  (note  that  here  y  ■  t-a).  We  should  also 
expect  good  performance  in  the  dynamic  problem  [cf.  Haji  et  a/.]  if  the 
marginal  costs  and  queue  size  are  such  that  the  linear  approximation  is  valid 
most  of  the  time.  In  the  many-link  case  we  have**  g'(t,y)  ■  ag(t.y)/at  > 
lym(b,/yt  +  Ky)  from  (3.17),  end  the  corresponding  rule  is 

^(t.y)  «  lym(b,  y)/r,  (3.28) 

with  y  as  in  (3.13);  in  the  special  case  of  linear  costs,  we  get  f(y)  *  fim Jr, 
again  a  time-independent  rule. 

If  we  specialize  to  constant  transmission  times  r,  we  can  find  optimal 
time-independent  rules  in  some  other  cases  of  interest.  Let  c^lR-IR  be  any 
CND  function,  with  derivative  m0:IR-IR,  and  consider  the  case  of  identical 
costs  c(b,t)  *  c  eft),  t  i  0,  in  a  single-queue  system.  In  this  case  the  optimal 
rule  is  ^(t.y)  *  y  (cf.  $3.2.3);  this  is  equivalent  to  FCFS 

*<y)  »  -a  (3.29) 

[cf.  Henderson  et  a/;  Thm.  2]  [cf.  Haji  et  a/],  and  thus  generates  an  optimal 
policy  for  the  G/D/1  queue.  To  show  this,  suppose  that  in  some  sequence  we 
have  a  arbitrary  packets  followed  by  packet  j  and  then  packet  i,  and  that 
y  i  i  y  y  If  we  interchange  i  and  j,  the  cost  of  packet  i  decreases  by 
c0(yi  *dr  *r)  -  c0(yi+fir)  and  that  of  packet  j  increases  by  c0(yj  *Ar*r)~ 
c0(y  j  +J8r);  the  net  increase  is  then 

yj-Hfl+Dr  y  ,♦<£«•  l)r 

/  m0(t)dt  -  /  m0(t)dt  S  0  (3.30) 

y  y  i4^’’ 


* ‘there  is  of  course  an  implicit  time-dependence  in  (3.14)  [and  similarly 
for  S(b,r)],  and  in  practice  might  depend  on  t  as  well;  we  assume  here  that 

these  parameters  do  not  change  significantly  over  large  numbers  of  transmis¬ 
sion  times. 


(for  every  r>0).  because  m0  is  nonnegative  and  nondecreasing.  In  fact,  it  is 
not  hard  to  show  that  if  |  r  i  “  r  j  |  £  for  all  i.j«Q,  then  the  rule 

makes  optimal  choices  in  the  case  of  variable  transmission  times  as  well. 

Now  in  a  somewhat  more  practical  vein,  we  can  derive  different  delay 
costs  from  c0  by  shifts  of  the  form  c(b,t)  ■  c^t^A*,)  -  t£0,  as  illus¬ 

trated  in  Fig.  6.  In  this  case  m(b.t)  ■  mo(t«-AJ,  and  it  is  not  hard  to  see  (or 
prove)  that  in  a  single  queue  the  optimal  rule  is  simply  f(t,y)  «  Ab  ♦  »,  or 

y)  ■  Ab  -  a;  (3.31) 

again  the  rule  generates  an  optimal  policy  for  the  G/D/1  queue.  Note  that  the 
due-date  scenario  may  be  expressed  as  the  particular  case  c^tt)  *  max{0,t}, 
with  Ab  *  -Tb  But  while  the  careful  construction  of  a  nominal  cost  c0  might 
yield  a  useful  collection  of  delay  costs  c(b,  ),  there  is  no  guarantee  of  this;  in 
particular  the  scaling  requirement  is  not  likely  to  be  met. 

In  the  many-link  case,  c(Ab)  is  time-independent,  and  we  may  assume 
g(t.y)  «  e0(i  ♦  r  +  Ab).  Then  if  .we  use  the  time-independent  future  estimate 
(3.15),  it  is  again  easy  to  see  that  the  rule 

f(  y)  ■  Ab*h-a  +  f  (3.32) 

is  optimal,  and  generates  an  optimal  policy  for  a  G/D/1  link.  However,  with 
the  more  realistic  time-dependent  future  estimate  (3.16),  we  have  g(t,y)  > 
Co(£yt*Ky  +Ab)  from  (3.17),  and  the  decision  costs  no  longer  correspond  to  a 
simple  family  of  shifted  functions.  In  particular  g'(t,y)  «  +  Ky  ♦  Ab); 

because  the  time-scaling  of  these  marginal  decision  costs  depends  on  y,  we  in 

*’We  can  improve  the  pessimistic  estimate  (3.15)  without  introducing  time 
dependence,  by  replacing  (t-a)  in  (3.16)  with  an  estimate  l(b,r)  of  the  queueing 
delay  on  the  given  link;  but  there  remains  a  tradeoff  between  the  quality  of 
the  estimate  and  our  ability  to  characterize  good  policies. 


general  cannot  make  the  same  kind  of  argument  as  in  (3.30),  and  cannot  find 
an  optimal  time-independent  rule.1,1 

In  general,  it  is  hard  to  say  much  about  the  nature  of  optimal  rules  when 
the  transmission  times  are  not  constant.  The  due-date  rule  ^(t,y)  «  -OVt)  is 
one  exception;  and  while  the  maximum  lateness  maxigtcIb^Sn)}  is  not  an  addi¬ 
tive  performance  measure,  it  is  appropriate  here  as  the  limiting  case 
{ZiJcfbfvSrJ]  ■“}  1/p  as  p-**.  With  a  time-independent  estimate  f.  the  form  of 
the  decision  costs  does  not  change  in  the  network  case;  thus  the  rule 

^y)  *  -Ut>  -  (h-a  +  f)],  (3.33) 

a  special  case  of  (3.32),  is  optimal  in  the  static  sense  for  variable  transmis¬ 
sion  times  as  well.  As  another  case  of  interest,  Schrage  [1973]  has  derived  a 
suboptimal  time-dependent  rule  for  variable  service  times  to  reduce  the  second 
moment  of  delay  [/'.«.  c(b,t)  ■  t2];  the  results  of  his  simulation  for  an  M/M/1 
queue  with  utilization  />  *  0.95  showed  substantial  improvement  over  FCFS.  In 
our  notation  the  rule  is 


^(t.y)  *  2|E2{t)/t  -  r2. 


(3.34) 


where  E(r)  is  the  expected  transmission  time  overall,  and  is  similarly  applica¬ 
ble  in  the  network  case  with  a  time-independent  estimate  f. 

Unfortunately,  we  have  not  found  reasonable  policies  for  the  important 
case  in  which  competing  cost  functions  have  very  different  forms.  Part  of  the 
problem  is  that  the  interchange  argument  is  not  reliable  in  general,  as 
suggested  earlier.  Now  consider  the  set  X  of  all  schedules  c  for  a  given  prob¬ 
lem  <\,o>.  The  success  of  the  interchange  argument  for  an  optimal  time- 
independent  rule  rests  on  the  fact  that  every  local  minimum  of  H  ,t,»):£-»IR  is 
a  global  minimum,  i.e.  T  is  in  some  sense  "convex"  in  £.  However  this  prop¬ 
erty  does  not  hold  in  general.  Conway  at  a/.  give  a  counterexample  in  which 
the  service  times  are  variable;  the  example  below  shows  that  the  property  can 
fail  for  constant  transmission  times  as  well. 

Suppose  that  packets  1,  2,  and  3  have  transmission  times  r,  and  arrive 
simultaneously  at  a  single  queue  system;  let  c(b,,t)  *  t  and  c(b2,t)  *  c(b3,t)  • 
amax{0.t-2T}.  with  «»1.  Then  the  sequences  1-2-3  and  1-3-2  each  have  total 
cost  ar,  and  are  local  minima;  in  either  case,  if  packet  1  is  interchanged  with 
the  succeeding  packet,  the  iost  increases  to  (c  *  «)r.  Yet  the  global  optima  are 
the  sequences  3-2-1  and  2-3-1  with  cost  2e.  Thus  a  selection  rule  may  be 
myopic  as  well,  in  the  sense  that  it  can  "paint  itself  into  a  corner"  by  not 
looking  at  the  entire  cost  function. 


CHAPTER  IV  -  SUMMARY  AND  CONCLUSION 


We  have  addressed  the  issue  of  delay  performance  in  packet  networks  by 
assigning  delay  costs  for  different  classes  of  traffic,  and  adopted  a  limiting* 
average  expected  cost  approach.  In  particular,  we  suggest  that  applications 
characterized  by  constraints  on  individual  packet  (or  round-trip)  response  times 
lead  to  strictly  convex  costs,  while  applications  for  which  job-related 
constraints  dominate  will  have  linear  costs.  We  showed  that  the  expected 
cost  in  equilibrium  is  an  equivalent  performance  measure,  and  characterized  the 
equilibrium  delay  performance  of  a  typical  network;  this  suggested  consider¬ 
able  potential  for  reducing  convex  costs  by  packet  scheduling,  especially  when 
link  traffic  is  relatively  heavy.  In  contrast,  routing  seemed  to  be  relatively 
inflexible  in  this  respect. 

A  distributed  scheduling  implementation  was  proposed,  which  incorporates 
knowledge  of  packet  histories  and  estimates  of  packet  futures;  we  know  little 
about  the  tradeoff  between  resolution  and  communication  overhead  associated 
with  time  stamps  for  packet  histories.  Link  scheduling  was  formulated  as  a 
dynamic  programming  problem,  with  decision  costs  representing  estimates  of 
the  delay  cost  on  system  exit.  We  concentrated  on  simple  static  selection 
rules  appropriate  in  a  real-time  environment,  but  showed  that  for  some  ideal¬ 
ized  cases  such  rules  are  optimal.  In  particular,  we  found  that,  in  the  case  of 
constant  transmission  times,  a  time-independent  priority  rule  which  is  optimal 
in  the  static  sense  always  generates  an  optimal  dynamic  policy.  The  most 
useful  case  we  found  was  that  of  shifted  delay  costs  of  the  form  c(b,t)  * 
c0(t  ♦  Ad),  where  c0  is  convex  and  nondecreasing;  in  this  case  the  rule  f{ y)  « 
Ab  ♦  h  -  a  ♦  f  is  optimal  for  a  G/D/1  link,  if  the  downstream  estimate  is 
time-independent.  We  did  not  find  useful  rules  for  the  important  case  in 


which  the  delay  costs  have  radically  different  forms,  but  helped  explain  why 
this  case  is  more  difficult. 

We  believe  that  the  basic  idea  potentially  has  practical  merit,  especially 
as  processing  costs  come  down  in  the  future. 


.V  V. 


APPENDIX 


A.1  Convergence  of  Averege  Cost  to  Equilibrium  Cost 

Theorem:  Suppose  s  sequence  {F^  of  distribution  functions  converges 
weekly  to  a  distribution  function  F,  and  let  c:  IR  -*  IR  be  nonnegative,  nonde¬ 
creasing,  and  continuous.  If 

lim  /  c(t)dFn(t)  »  0  uniformly  in  n,  (A.1) 

A-»o  I  t  I  >A 


then 


lim  (1/N)  I  /c(t)dF^t)  «  /c(t)dF(t)  (A.2) 

Nt  n*  1 

Remarks:  Note  that  if  Fn  is  nonnegative  [/.*.  Fn(t)  *  0  for  t<0]  for  all  n. 
then  so  is  F,  and  we  are  only  concerned  about  the  behavior  of  c  on  [0,«). 
Thus  if  we  substitute  F  j {  ;b)  for  F^  F(  ;b)  for  F,  and  c(b,  )  for  c,  (A.2)  is 
equivalent  to  (2.14). 

In  fact  it  is  easy  to  show  that  if  /c(t)|t|pdFn(t)  is  uniformly  bounded  for 
some  p>0,  then  (A.1)  holds  [use  the  approach  on  p.  186  of  Loeve];  thus 
uniform  boundedness  of  the  expected  costs  E{c[b,s(nj)]}  is  "almost"  suffi¬ 
cient  for  (2.14). 

Proof:  Denote  by  Cn  and  C  the  expectations  with  respect  to  Fn  and  F  in 
(A.2);  we  first  show**  that  Cn-  C.  Define  for  every  A  f  IR  the  function  cA  by 


’'This  is  a  known  result  [Loeve;  $11.4];  the  proof  here  is  based  on  a 
moment  convergence  proof  from  Chung  [Thm.  4.5.2],  and  is  more  direct. 


c(t)  if  tSA 

c*(t)  ■  { 

c<A)  if  t>A 
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These  functions  are  all  continuous  and  bounded  on  IR,  so  we  have  for  all  A 


lim  /cA(t)dF  n(t)  «  /c  A(t)dF(t) 

nt 


directly  from  the  definition  of  weak  convergence  [Ash;  $$4.5,  8.1].  Also. 


/[c(t)-cA(t)]  dF  At)  -  /A[c(t>-cA(t)]dF^t) 


S  /  c(t)dF^t)  <  /  c(t)dF  n(t), 
t>*  m  >a 


so  by  (A.1),  /( c-c*)dFn  vanishes  uniformly,  /.*. 


Mm  /cA(t)dF  „(t)  «  Cn 


l — «, 


uniformly  in  n. 

It  then  follows  from  (A.4)  and  (A.6),  and  properties  of  uniform  conver¬ 
gence  [cl.  Rudin;  p.  149],  that  ,TcAdF  converges  and 


lim  /cA(t)dF(t)  *  lim  Cn, 

A  I  n  I 


i.e.  the  iterated  limits  exist  and  are  equal.  But  /cAdF  -  C  by  (essentially) 
monotone  convergence  [Ash;  p.  44] ,  so  C  n  -  C. 

Thus  for  every  c>0,  there  exists  an  M  such  that  |Cn-C|  <  c/2  for  n2M. 


le  r  max{(2/c)  Z(Cn-C),  M}1 

n*  1 


-  >v- 


•  V 
O  ' 


TTrrr  i.v.-  >■.■  rrc- 


':v.v:v 
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where  fxl  is  the  smallest  integer  not  less  than  x;  then  for  N£L.  we  have 
|  { 1  /N)  IC„  -  C| 

n«l 


«  ||1/N)n|jC„-C)| 

«  |(1/N)J(C„-C>  ♦  (1/N)  J|Cn-C)| 

S  |{1/L)MZ{C„-C)|  *  |(N-M*1)- 1  Z(Cn-C)| 

n* i  n*M 

£  (c/ 2)  ♦  (c/2)  «  c,  (A.9) 

i.e.  (1/N)ZnC  n  -*  C,  which  is  equivalent  to  (A.2). 


A.2  Alternate  Route  Delays  in  an  Optimal  Flow  Assignment 

We  consider  here,  in  a  network  setting,  the  traffic  leaving  a  given  node, 
intended  for  a  given  destination;  two  alternate  paths  p,  and  p2,  of  Ni  and  N2 
hops,  are  aveilable.  We  model  the  equilibrium  behavior  with  a  function  »,  such 
that  v(f)  gives  the  expected  number  of  packets  (in  queue  and  transmitter)  on  a 
link  with  flow  f,  independently  at  all  links  in  the  network.  The  expected 
number  of  packets  in  the  system  is  then  the  sum  L  ■  Z  e v(f  •>  over  all  links 
ecE.  and  by  Little’s  formula  [ cf .  Stidham,  1972;  pp.  1122-23]  we  have  L  ■  XS. 
where  X  is  the  total  arrival  rate.  Thus  a  minimum-delay  flow  assignment  is 
one  which  minimizes  IEv(fJ;  here  we  examine  some  of  the  implications  of 
such  an  assignment. 

If  we  assume  further  that  *  is  convex  (as  it  is  for  M/G/1),  we  can  charac¬ 
terize  the  optimal  flow  assignment  as  follows  [Bertsekas,  1981;  eqn.  (4.4)  and 
below]:  if  v'(f)  ■  a»(f)/af,  and  we  define  the  first  derivative  length  (FDL)  of  a 
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path  p  by  Xp*'(f«)>  then  path  flow  is  positive  only  on  paths  with  minimum  FDL 
Thus  if  we  have  positive  flow  on  both  p,  and  p2,  optimality  requires  that 

di  ■  X  *VJ  «  X  r'<f  J  ad*  (A.  10) 

PI  P2 

Intuitively,  if  p^  had  a  smaller  FDL,  then  some  flow  could  be  shifted  from  p2 
to  pv  and  by  convexity  the  sum  *  Xp2»(f.)  would  decrease,  equiv¬ 

alent  to  a  decrease  in  overall  delay. 

If  the  derivative  •>'  is  convex  as  well  (also  true  for  M/G/1),  we  expect 
that,  for  a  given  first  derivative  length,  variability  among  link  flows  along  a 
path  implies  a  smaller  average  link  flow  (over  the  path)  than  does  a  relatively 
constant  link  flow  profile.  But  because  r  itself  is  convex,  variability  among 
(ink  flows  also  requires  less  average  link  flow  for  a  given  number  of  packets 
on  the  path,  and  when  Nt  «  N2  these  factors  tend  to  cancel.  As  an  example, 
a  string  of  four  M/M/1  links,  with  expected  transmission  times  E(r),  capacities 
0.  and  flows  0.4 0.  0.50,  0.60,  and  0.70,  has  a  path  delay  of  Si  =  9.5E(r);  the 
same  system  has  a  path  delay  of  S2  ~  9.8E(r)  if  all  link  flows  are  equal  and 
adjusted  such  that  the  FDls  are  equal,  d2  «  dv 

If  N,  *  N2,  the  path  delays  are  not  as  constrained.  In  the  case  of  M/M/1 
links  with  constant  path  flows  ^  and  f2,  optimality  requires 

hMM-f,)3  »  Na0/(0-f2)2,  or  (A.l  1) 

i/(0-f  i)  «  '✓N  2/(0-f  a),  which  gives 

N,E(fl)VN2/(0-f  0  «  N2E(fi)VN1/(0-f2l,  or 

SiVN2  «  Sa^N,.  (A.  12) 

where  E (fl)  is  the  expected  packet  length.  Thus  the  greater  hop-length  path  has 
a  larger  path  delay,  but  the  relative  difference  is  still  not  so  large  for  N2 


close  to  Nt.  The  previous  psregrsph  suggests  thet  non-constant  path  flows 
should  not  significantly  alter  this  result. 

A.3  Optimal  Priority-based  Policies  for  G/D/1  Links 

To  formalize  the  G/D/1  case,  let  Ye  ■  {*« T:  r(i)  >  r(j)  for  all  i,j  t  Q}  be 
the  set  of  all  queue  lists  with  constant  transmission  times.  Then  given  g,  a 
priority  rule  fm  is  optimal  in  the  case  of  constant  transmission  times  if  it 
generates  an  optimal  sequence  for  every  <t,»»  t  tP^Ye  (it  is  intuitively  compel¬ 
ling  that  if  a  rule  is  optimal  for  any  given  constant  r,  then  it  is  optimal  for 
every  such  constant;  indeed,  we  show  in  $3.4  the  optimality  of  some  rules  of 
this  type,  and  the  proofs  are  always  independent  of  r).  Now  if  Qc  ■  («cQ: 
r  ,(<•)  *  r }(»)  for  all  i,j  t  IN*}  is  the  set  of  all  realizations  with  constant  trans¬ 
mission  times,  we  define  and  J(*|0^  by  conditioning  the 

expectations**  in  (3.10)  and  (3.20)  on  ( l &  Then  we  have  the  following: 

Theorem:  Given  a  decision  cost  g,  suppose  there  exists  a 
time-independent  priority  function  f  *  which  is  optimal  in  the  case  of  constant 
transmission  times.  Then  for  every  P,  in  the  problem  <P,g>  the  stationary 
policy  »*  ■  *,...>  generated  by  f*  satisfies 

JN(**|0C)  <  JN(i|0e)  (A.  13) 

for  every  *<!l  Furthermore,  if  J(**|0J  exists  end  is  finite,  then 

J(» *|0c>  S  liminf  (1/N)  {JnH0c>  4  E CT }  (A.14) 

N  I 

for  every  aril,  where  T*^#)  abbreviates  r*  [t^*),*^*)]. 

••If  necessary,  we  can  formalize  the  expectations  by  simply  considering  a 
new  probability  measure  Pc  associated  with  Oe- 


Proof:  Consider  the  problem  of  minimizing 

j (.i***)  *  i  <a.is) 

k«0 

over  sell,  where  r*L(«,#)  ebbreviates  T*  [tL(i*,*),ti L(a,»)];  jj.  is  the  incurred  cost 
plus  liability  after  L  decisions,  for  a  given  realization  (assuming  no  further 
arrivals).  We  show  below  that,  in  fact, 

jL(**0  -  i (A.  16) 

for  every  •  £0C.  »«n,  L  Now  in  the  finite  case,  QN  is  empty  for  every  «,  », 
so  T^t,***!)  ■  0;  then  with  L  ■  N  in  (A.16),  taking  expectations  conditioned  on 
0C  preserves  the  inequality  [cf.  Ash;  p.  41],  giving  (A.13).  In  the  infinite  case, 
take  expectations  in  (A.16)  conditioned  on**  divide  by  L,  and  take  lim  inf 
as  L-»,  each  of  which  preserves  the  inequality;  this  gives  (A.14)  when  the  LHS 
converges. 

Now  let  «eGc  and  L  be  given,  end  suppose  that  rn{«)  «  r  for  all  n:  to 
get  (A.16),  we  show  that  for  every  policy  »°ell,  there  exists  a  sequence  of 
policies  *°,  #  \  ...  #L"\  •*,  in  which  ju  is  non-increasing,  Le.  j  i 

’)  >  ...  i  ji>,#*).  To  see  how  jL  changes  from  one  policy  to  the  next,  it 
is  helpful  to  visualize  the  packet  selections  u0,  u„  ...  uL.1  (dictated  by  a 
given  policy)  laid  out  along  the  time  axis  at  epochs  to  t1#  ...  tL- 1  (note  that 
for  constant  transmission  times,  the  tk  ere  independent  of  the  policy,  for  a 
given  realization);  follow  these  by  the  packets  in  Qu  laid  out  according  to  the 
optimal  static  sequence  c *  generated  by  f \  at  epochs  tL,  tL+r,  ...  tu*(q-1)r 
(where  q  «  |QL|  is  also  independent  of  the  policy).  The  basic  idea  is  that,  if 

**Note  that  we  could  just  as  easily  have  conditioned  on  a  given  initial 
state  here  to  get  the  more  standard  dynamic  programming  statement. 
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the  packet  Uj  selected  at  tj  under  the  policy  *J  is  not  the  one  dictated  by  p*. 
i.e.  if  Ujff*)  *  /»*[•» J)].  then  we  find  a  new  policy  under  which  the 

positions  of  packets  Uj(»J)  and  in  the  ordering  are  interchanged;  the 

effect  of  the  transformations  is  that  policy  has  the  form 

.. jt* . /i* jf p* }  +  ),...>  (j «0, 1....L-1).  Note  that  every  such  interchange  must 
occur  within  some  busy  period  {because  p*  is  admissible);  the  lemma  below 
shows  that  no  such  interchange  can  increase  the  value  of  jL  (the  lemma  is 
proved  after  the  theorem). 

Lemma:  Given  a  decision  cost  g,  suppose  there  exists  a  time- 

independent  priority  rule  which  is  optimal  in  the  case  of  constant  transmis¬ 
sion  times;  let  p*  be  the  map  which  defines  the  policy  generated  by  f‘.  Then 
if  «cYc  is  a  queue  list  with  transmission  times  r,  p*{v)  *  i  t  Q  implies 


g[t,»(i)]  ♦  g[t*(m*1)r.  »(j)]  <  g[t,*{j)]  ♦  g[t*(m+1)r,  »(i)] 


,  (A.  17) 


for  every  i.j  c  Q,  telR,  m  «  0,  1 .  Thus  if  packets  i  and  j  are  separated  by 

m  packets  in  an  ordering,  the  combined  cost  of  both  packets  is  no  greater 
with  packet  i  in  the  earlier  position. 

Formally,  we  get  »J  +  1  from  wi  as  follows,  for  j-l  «=  1,  ...  L-1: 

Case  A:  ><*[#j(»J)]  *  u  j(ir j);  i.e.  policy  vi  makes  the  choice  dictated  by 
P*  at  epoch  tj,  so  simply  pick  »■ J * 1  *  irJ,  and  jL  does  not  change. 

Case  B:  /»* t » j(»J)3  *  Up(»j)  for  some  p  e  {j*1,...L-1};  i.e.  the  packet 
which  /»*  would  have  chosen  at  tj  is  instead  selected  at  some  later  epoch 
tp  <  tL  by  *  K  Then  pick  any  such  that 


-  V  o  * 
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/r^’plxp U^’8  « 

•  /•1k[xk(*J>].  k#j.p  (A.  18) 

(thus  #J  +  1  is  not  uniquely  specified  here).  Some  thought  shows  thet  these  ere 
all  admissible  choices  if  • i  is  admissible.  Cleerly  the  packets  in  QL  are  not 
affected  by  this  interchange,  so  r*L(*J,M)  ■  r*L(»4);  similarly,  gk(»J,f1)  ■  gK(» ■*) 
for  kSL-1,  k*»j,p.  Only  the  costs  of  packets  Uj(»J)  and  Up(»4)  are  affected,  and 
by  the  lemma  (with  m+1  *  p-j)  we  have 

g(tj,y[uj(*i+1)]  ♦  gitp.ytup^1*1)]) 

s  gft  j.y[uj(fJ)]}  ♦  g{tp,y[urf#4)]}.  (A.19) 

Thus  j,>4  +  1)  S  jL(,i). 

Case  C:  ?'[*](**}]  eQL(*J);  i.e.  the  packet  that  would  have  been 
selected  at  tj  by  f  is  not  chosen  by  oi  in  the  first  L  decisions.  In  this  case 
pick  any  which  satifies 

V  [Xk(*^+  *)3  •  /iJk[xK(xJ)3.  k#i.  (A.20) 

Now  if  **  is  the  (optimal)  sequence  for  »L(»4)  generated  by  f',  let  o'  be  the 
sequence  for  *L(*4'M)  obtained  by  substituting  Uj(»J)  for  /»*[»j(»J)]  in  o';  then 
assuming  a  packets  proceed  p'[t  in  o'  [or  uj(ir J)  in  o’],  the  lemma 

gives  (with  m*l  ■  t-l+tl) 

g{tj»yCuj<»J*1)3}  *  g{tu^r,y[uJ<»J)]} 


2  g{tj,y[Uj(f J)]}  ♦  gftL*ar,y[u,(.r 4  ♦’)]}. 


(A.21) 


Since  9k(fJ  +  1)  *  9k(*J)  for  kSL-1.  kPj,  and  the  costs  g(tL+|*  n|r,  yn)  for  aff 
other  packets  ncQu,  rw*/i*[»  j(*J)],  are  unaffected  by  the  interchange,  we  have 

k«0  k«0 

iL*I  gkU1)  ♦  r[#*,tL.a  ,.(»*)]  (A.2 

k«0 

(where  the  first  inequality  follows  from  the  definition  of  I**).  or  in  other 
words  jL(#J  +  1)  <  jL(*J). 

We  get  »*  from  #L_1  in  essentially  the  same  way,  in  each  case,  except 
that  for  k*L, L+1, ...  we  must  of  course  set  /i*k  *  ?’•  which  clearly  has  no 
effect  on  jL. 

Proof  of  Lemma:  The  lemma  is  a  statement  about  only  packets  i  and  j; 
to  prove  it  we  define  a  new  queue  list  *'cYc  consisting  of  packets  i  and  j, 
and  m  other  packets  whose  characteristics  are  the  same  as  those  of  j.  For 
concreteness,  let  n0  «  max{i,j}  and  0'  ■  {i,  j, n0,...no+m-l};  then  »'(i)  ■  *(i), 
and  #'(n)  a  *Kj)  for  n  ■  j,  n^  ...  no+m-l. 

Now  consider  the  sequence  defined  by 

*(j)  <  eOiJ  <  ...  <  ••(no+m-l)  <  *(i).  (A.23 

The  first  two  packets  in  the  sequence  (j  and  no+m-1)  are  indistinguishable,  and 
interchanging  them  clearly  cannot  affect  the  total  cost  IV,t,»');  similarly  we 
can  bubble  packet  j  to  the  next-to-last  position  for  the  seme  total  cost.  Now 
we  can  interchange  packets  i  and  j  without  increasing  the  total  cost:  simply 
note  that  i  precedes  j  in  the  optimal  sequence  for  the  corresponding 
two-pecket  problem.  Iterating  the  same  argument,  we  can  bubble  packet  i  to 
the  first  position  without  increasing  the  total  cost.  Finally,  packets 
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